Software Center and Highly Configurable Robotic Systems for Surgery and Other Uses

ABSTRACT

Telerobotic, telesurgical, and/or surgical robotic devices, systems, and methods employ surgical robotic linkages that may have more degrees of freedom than an associated surgical end effector in space. A processor can calculate a tool motion that includes pivoting of the tool about an aperture site. Linkages movable along a range of configurations for a given end effector position may be driven toward configurations which inhibit collisions. Refined robotic linkages and methods for their use are also provided.

CROSS-REFERENCE TO RELATED APPLICATIONS

The present application is a Continuation of Ser. No. 14/265,671 filedApr. 30, 2014 (Allowed); which is a Continuation of Ser. No. 13/175,590filed Jul. 1, 2011 (now U.S. Pat. No. 8,749,190); which is aContinuation of Ser. No. 11/133,423 filed May 19, 2005 (now U.S. Pat.No. 8,004,229); the full disclosures which are incorporated herein byreference in their entirety for all purposes.

STATEMENT AS TO RIGHTS TO INVENTIONS MADE UNDER FEDERALLY SPONSOREDRESEARCH OR DEVELOPMENT

NOT APPLICABLE

BACKGROUND OF THE INVENTION

The present invention generally provides improved surgical and roboticdevices, systems, and methods.

Minimally invasive medical techniques are aimed at reducing the amountof extraneous tissue which is damaged during diagnostic or surgicalprocedures, thereby reducing patient recovery time, discomfort, anddeleterious side effects. Millions of surgeries are performed each yearin the United States. Many of these surgeries can potentially beperformed in a minimally invasive manner. However, only a relativelysmall number of surgeries currently use these techniques due tolimitations in minimally invasive surgical instruments and techniquesand the additional surgical training required to master them.

Minimally invasive telesurgical systems for use in surgery are beingdeveloped to increase a surgeon's dexterity as well as to allow asurgeon to operate on a patient from a remote location. Telesurgery is ageneral term for surgical systems where the surgeon uses some form ofremote control, e.g., a servomechanism, or the like, to manipulatesurgical instrument movements rather than directly holding and movingthe instruments by hand. In such a telesurgery system, the surgeon isprovided with an image of the surgical site at the remote location.While viewing typically a three-dimensional image of the surgical siteon a suitable viewer or display, the surgeon performs the surgicalprocedures on the patient by manipulating master control input devices,which in turn control the motion of robotic instruments. The roboticsurgical instruments can be inserted through small, minimally invasivesurgical apertures to treat tissues at surgical sites within thepatient, often the trauma associated with accessing for open surgery.These robotic systems can move the working ends of the surgicalinstruments with sufficient dexterity to perform quite intricatesurgical tasks, often by pivoting shafts of the instruments at theminimally invasive aperture, sliding of the shaft axially through theaperture, rotating of the shaft within the aperture, and/or the like.

The servomechanism used for telesurgery will often accept input from twomaster controllers (one for each of the surgeon's hands) and may includetwo or more robotic arms or manipulators. Mapping of the hand movementsto the image of the robotic instruments displayed by the image capturedevice can help provide the surgeon with accurate control over theinstruments associated with each hand. In many surgical robotic systems,one or more additional robotic manipulators are included for moving anendoscope or other image capture device, additional surgicalinstruments, or the like.

While the new robotic surgical systems and devices have proven highlyeffective and advantageous, still further improvements would bedesirable. For example, when moving the surgical instruments within aminimally invasive surgical site, robotic surgical manipulators mayexhibit a significant amount of movement outside the patient,particularly when pivoting instruments about minimally invasiveapertures. As the instruments may independently pivot about theirassociated apertures at the same time, the robotic manipulators disposedoutside the patient may sometimes collide with each other (or with otherstructures or personnel). Additionally, set-up of the surgical roboticsystem in preparation for surgery can be challenging, and re-configuringthe system to access different tissues of the patient during differentphases of a procedure can be inconvenient.

For these and other reasons, it would be advantageous to provideimproved devices, systems, and methods for surgery, robotic surgery, andother robotic applications. It would be particularly beneficial if theseimproved technologies provided a faster and easier set-up, and/orinhibited collisions of the robotic devices during use. Ideally, theseimprovements would be provided without significantly increasing thesize, mechanical complexity, or costs of these systems, and whilemaintaining or improving their dexterity.

BRIEF SUMMARY OF THE INVENTION

The present invention generally provides improved robotic and/orsurgical devices, systems, and methods. In many embodiments, theinvention will employ highly configurable surgical robotic manipulators.These manipulators, for example, may have more degrees of freedom ofmovement than the associated surgical end effectors have within asurgical workspace. The present invention will often include a processorconfigured by software instructions to calculate a motion of the roboticlinkage that includes pivoting a shaft of the manipulator linkage aboutan aperture site hence the notion of software centering. The joints ofthe robotic manipulators supporting the end effectors can allow themanipulator to move throughout a range of different configurations for agiven end effector position, in some cases even when constraininglateral movement of the shaft from the aperture site so as to avoidinjury to the adjacent tissues. The processor can take advantage of suchmanipulators to drive the manipulators to configurations which inhibitcollisions involving one or more moving robotic structures. Set-up ofsuch highly configurable robotic manipulators can be facilitated byprocessors which drive one or more joints of the manipulators while themanipulator is being positioned manually by the system operator (or bysome other external interaction), with the joints optionally beingdriven in response to movements of other joints along the kinematicchain of the manipulator. Embodiments can adjust a center of pivotalmotion of the manipulator in response to patient breathing and/ormovement, in some cases by sensing forces applied between themanipulator and the tissues along the aperture site. Refined roboticstructures for use in minimally invasive surgical applications and otherapplications are also provided, along with related robotic methods.

In one aspect of the present invention, a redundant degrees of freedom(RDOF) surgical robotic system with manipulate input is provided. TheRDOF surgical robotic system comprises a manipulator assembly, an inputdevice, and a processor. In response to a command to effect a desiredmovement of the end effector as received by the input device, themanipulator assembly manipulates a distal end effector relative to aproximal base. The manipulator has a plurality of joints providingsufficient degrees of freedom to allow a range of joint states for anend effector position. The processor couples the input device to themanipulator assembly and may be configured to operate in differentmodes. In the end effector manipulation mode, the processor determinesmovements of the joints in response to the command so as to move the endeffector with the desired movement. In the clutch mode, the processordrives at least one of the joints in response to an externalarticulation of another joint of the manipulator assembly. The clutchmode may be a pose clutch mode, an instrument clutch mode, or a portclutch mode.

In another aspect of the present invention, a RDOF software centeringsurgical robotic system is provided. The RDOF software centeringsurgical robotic system comprises a processor, an input device, amanipulator, and a surgical instrument. The surgical instrument has aproximal end, a distal end effector suitable for insertion into apatient, and an intermediate portion therebetween. The manipulatorsupports the proximal end of the instrument and is therefore capable ofmoving/controlling the instrument from outside the patient. Between themanipulator and the instrument, there are a plurality of driven jointsproviding sufficient degrees of freedom to allow a range of joint statesfor an end effector position when the intermediate portion of the endeffector passes through an access site. The input device receives acommand to effect a desired end effector's movement. The processorcouples the input device to the manipulator. In response to thecommanded movement, the processor determines movements of the joints sothat the intermediate portion of the instrument is within the accesssite during the end effector's desired movement.

In one embodiment, the movements of the joints as determined by theprocessor may also be designed to inhibit manipulator collision. Inanother embodiment, the movements of the joints as determined by theprocessor may be designed to drive the manipulator to desiredcombinations of joint states that achieve the underconstrained primarysolution and a secondary control task. In yet another embodiment, themovements of the joints as determined by the processor may be designedto inhibit movement of a pivotal center of the intermediate portion ofthe instrument in response to a port stiffness factor. In yet anotherembodiment, the movements of the joints as determined by the processorcomply with a priority task selected from a priority list. In yetanother embodiment, the RDOF software centering surgical robotic systemfurther include a sensor system to indicate to the processor a positionof the access site and/or a reactive force between the intermediateportion of the instrument and the access site aperture.

In yet another aspect of the present invention, a multi-manipulator RDOFsurgical robot with collision avoidance capability is provided. Themulti-manipulator RDOF surgical robot comprises a first manipulatorassembly, a second manipulator assembly, an input device, and aprocessor. The first manipulator assembly has a first end effector and aplurality of joint states for one first end effector position. Thesecond manipulator assembly has a second end effector and has thecapability to transmit state signals indicating movement of the secondmanipulator assembly. The input device receives an input for a firstdesired movement of the first end effector. The processor is coupled tothe input, the first manipulator assembly, and the second manipulatorassembly. The processor determines a movement of the first manipulatorassembly in response to the input but with the second manipulatorassembly state signals being taken into consideration so as to inhibitcollisions between the manipulator assemblies. Using the determinedmovement, the processor controls the first end effector to effect thefirst desired movement.

In yet another aspect of the present invention, a surgical roboticmanipulator with an upper arm roll is provided. The surgical roboticmanipulator comprises a moveable surgical instrument holder, a basewhich is positionable near a workspace, and an arm assembly which ispivotally coupled between the base and instrument holder. The armassembly includes a first link having a first axis, a second link havinga second axis, a pivotal arm joint coupled between the first link andthe second link to vary an angle between the first axis and the secondaxis, and a first roll joint between the base and the pivotal arm joint.The first roll joint has an arm roll axis extending along the firstaxis.

A further understanding of the nature and advantages of the presentinvention will become apparent by reference to the remaining portions ofthe specification and drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a perspective view illustrating a master surgeon console orworkstation for inputting surgical procedure commands, the consoleincluding a processor for generating manipulator command signals inresponse to the input commands.

FIG. 1B is a side view schematically illustrating a surgical stationhaving a plurality of robotic manipulators for robotically movingsurgical instruments having surgical end effectors at an internalsurgical site within a patient, and a method for performing a surgicalprocedure with the manipulator-instrument assemblies.

FIG. 1C is a perspective view schematically illustrating a surgicalmethod performed at a minimally invasive surgical site within a patientusing the manipulator assemblies and surgeon console of FIGS. 1A and 1B.

FIG. 2 is a side view illustrating a master controller input devicehaving a handle for inputting a desired movement to the workstation ofFIG. 1A.

FIG. 3 is perspective view of an exemplary robotic surgical instrumentor tool to be mounted on the manipulators of FIG. 1B.

FIGS. 3A-3F are perspective views of a plurality of different endeffectors for surgical instruments of different types.

FIGS. 4A-4C are bottom, side, and back views of an exemplary roboticmanipulator assembly having a range of joint states for a given endeffector position.

FIGS. 5A and 5B are side views of the manipulator of FIGS. 4A-C, andillustrate a downwardly oriented elbow joint configuration and anupwardly oriented elbow joint configuration of the manipulator for agiven end effector position.

FIG. 6 schematically illustrates the degrees of freedom provided by themanipulator of FIGS. 4A-C and the instrument of FIG. 3 mounted thereon.

FIG. 7A schematically illustrates a manipulator assembly insertedthrough an aperture.

FIG. 7B schematically illustrates some of the challenges in manuallyrepositioning the highly configurable manipulator assembly of FIG. 7A toa new aperture position.

FIG. 7C schematically illustrates reconfiguring of the arm of FIG. 7A soas to enhance range of motion or the like during manual repositioning ofthe manipulator to a new aperture position.

FIGS. 8A-8D schematically illustrate different clutching modes formanually repositioning at least a portion of a robotic manipulatorassembly to accommodate a new aperture position, to pivot an instrumentholder about an aperture, to change a pose of the manipulator withoutmoving the end effector or instrument holder, and the like.

FIG. 8E schematically illustrates a highly configurable manipulatorassembly having a clutch input switch so as to facilitate manualpositioning of a surgical instrument adjacent a minimally invasiveaperture while a processor configures the manipulator joint in responseto the manual movement.

FIGS. 9A and 9B schematically illustrate robotically reconfiguring ofthe joints of the manipulator assembly within a range of alternativejoint configurations during manual movement of the arm.

FIG. 10A is a simplified block diagram schematically illustrating afully constrained inverse Jacobian master/slave velocity controller.

FIG. 10B is a simplified diagram of a modified master/slave controllerin which an inverse Jacobian controller module is combined with a secondmodule having a configuration dependent subspace filter so as to allowcontrol over a manipulator which is overconstrained or underconstrained.

FIG. 10C illustrates a refinement of the simplified master-slave controlillustrated in FIG. 10B.

FIG. 11 schematically illustrates an exemplary inverse Jacobiancontroller for a fully constrained master/slave robotic surgical system.

FIG. 12 schematically illustrates a modified portion of the controllerof FIG. 11, in which the inverse Jacobian controller has been modifiedwith a configuration dependent filter so that the controller respectspriority of differing levels of system constraints and/or goals.

FIG. 13 schematically illustrates a slave manipulator assembly having acannula force sensor.

FIG. 13A schematically illustrates a hybrid force/position controllerfor use with the manipulator assembly of FIG. 13 or other manipulatorsystems described herein.

FIGS. 13B and 13C schematically illustrate aspects of the control systemof FIG. 13A.

FIGS. 14A and 14B schematically illustrate an instrument coordinateframe and an instrument shaft moving pivotally about a minimallyinvasive access site, respectively.

FIG. 15 is a schematic of an alternative control system of theworkstation processor in which embodiment of the present invention maybe implemented.

FIG. 16 schematically illustrates a modified control system with modulesfor solving for joint motions of manipulators having excess degrees offreedom, as suitable for modifying the system of FIG. 15 so as toimplement an embodiment of the invention.

FIG. 17 schematically illustrates an inverse kinematic solutioncalculation as performed in one or more control module of FIG. 16.

FIG. 18A illustrates exemplary linkage assembly geometry and a preferredjoint configuration for the linkage assembly.

FIG. 18B is a top view of the manipulator assembly of FIG. 18A, andillustrates a planar configuration of the manipulator assembly.

FIGS. 18C and 18D are top views illustrating planar poses for adjacentmanipulator assemblies in a multi-manipulator slave system.

FIGS. 18E and 18F illustrate a manipulator assembly and show a desiredrange of motion for one of the joints of the assembly.

FIG. 18G is a perspective view showing an exemplary collision sensorsystem along a surface of a manipulator assembly.

DETAILED DESCRIPTION OF THE INVENTION

The present invention generally provides improved surgical and roboticdevices, systems, and methods. The invention is particularlyadvantageous for use with surgical robotic systems in which a pluralityof surgical tools or instruments will be mounted on and moved by anassociated plurality of robotic manipulators during a surgicalprocedure. The robotic systems will often comprise telerobotic,telesurgical, and/or telepresence systems that include processorsconfigured as master-slave controllers. By providing robotic systemsemploying processors appropriately configured to move manipulatorassemblies with articulated linkages having relatively large numbers ofdegrees of freedom, the motion of the linkages can be tailored for workthrough a minimally invasive access site. The large number of degrees offreedom may also allow a processor to position the manipulators so as toinhibit interference or collisions between these moving structures, andthe like.

The robotic manipulator assemblies described herein will often include arobotic manipulator and a tool mounted thereon (the tool oftencomprising a surgical instrument in surgical versions), although theterm “robotic assembly” will also encompass the manipulator without thetool mounted thereon. The term “tool” encompasses both general orindustrial robotic tools and specialized robotic surgical instruments,with these later structures often including an end effector which issuitable for manipulation of tissue, treatment of tissue, imaging oftissue, or the like. The tool/manipulator interface will often be aquick disconnect tool holder or coupling, allowing rapid removal andreplacement of the tool with an alternate tool. The manipulator assemblywill often have a base which is fixed in space during at least a portionof a robotic procedure, and the manipulator assembly may include anumber of degrees of freedom between the base and an end effector of thetool. Actuation of the end effector (such as opening or closing of thejaws of a gripping device, energizing an electrosurgical paddle, or thelike) will often be separate from, and in addition to, these manipulatorassembly degrees of freedom.

The end effector will typically move in the workspace with between twoand six degrees of freedom. As used herein, the term “position”encompasses both location and orientation. Hence, a change in a positionof an end effector (for example) may involve a translation of the endeffector from a first location to a second location, a rotation of theend effector from a first orientation to a second orientation, or acombination of both. When used for minimally invasive robotic surgery,movement of the manipulator assembly may be controlled by a processor ofthe system so that a shaft or intermediate portion of the tool orinstrument is constrained to a safe motion through a minimally invasivesurgical access site or other aperture. Such motion may include, forexample, axial insertion of the shaft through the aperture site,rotation of the shaft about its axis, and pivotal motion of the shaftabout a pivot point adjacent the access site, but will often precludeexcessive lateral motion of the shaft which might otherwise tear thetissues adjacent the aperture or enlarge the access site inadvertently.Some or all of such constraint on the manipulator motion at the accesssite may be imposed using mechanical manipulator joint linkages whichinhibit improper motions, or may in part or in full be imposed usingrobotic data processing and control techniques. Hence, such minimallyinvasive aperture-constrained motion of the manipulator assembly mayemploy between zero and three degrees of freedom of the manipulatorassembly.

Many of the exemplary manipulator assemblies described herein will havemore degrees of freedom than are needed to position and move an endeffector within a surgical site. For example, a surgical end effectorthat can be positioned with six degrees of freedom at an internalsurgical site through a minimally invasive aperture may in someembodiments have nine degrees of freedom (six end effector degrees offreedom—three for location, and three for orientation—plus three degreesof freedom to comply with the access site constraints), but will oftenhave ten or more degrees of freedom. Highly configurable manipulatorassemblies having more degrees of freedom than are needed for a givenend effector position can be described as having or providing sufficientdegrees of freedom to allow a range of joint states for an end effectorposition in a workspace. For example, for a given end effector position,the manipulator assembly may occupy (and be driven between) any of arange of alternative manipulator linkage positions. Similarly, for agiven end effector velocity vector, the manipulator assembly may have arange of differing joint movement speeds for the various joints of themanipulator assembly.

The invention provides robotic linkage structures which are particularlywell suited for surgical (and other) applications in which a wide rangeof motion is desired, and for which a limited dedicated volume isavailable due to the presence of other robotic linkages, surgicalpersonnel and equipment, and the like. The large range of motion andreduced volume needed for each robotic linkage may also provide greaterflexibility between the location of the robotic support structure andthe surgical or other workspace, thereby facilitating and speeding upsetup. Hence, while the most immediate applications for the presentinvention may include telesurgical systems, the structures, devices, andsystems described herein may also find applications in a wide variety ofother telerobotic and robotic applications.

The term “state” of a joint or the like will often herein refer to thecontrol variables associated with the joint. For example, the state ofan angular joint can refer to the angle defined by that joint within itsrange of motion, and/or to the angular velocity of the joint. Similarly,the state of an axial or prismatic joint may refer to the joint's axialposition, and/or to its axial velocity. While many of the controllersdescribed herein comprise velocity controllers, they often also havesome position control aspects. Alternative embodiments may relyprimarily or entirely on position controllers, acceleration controllers,or the like. Hence, so long as the movements described are based on theassociated calculations, the calculations of movements of the joints andmovements of an end effector described herein may be performed using aposition control algorithm, a velocity control algorithm, a combinationof both, and/or the like.

Embodiments of the invention may include a processor which is configuredto take advantage of the degrees of freedom of a manipulator structurefor a plurality of tasks, goals, or purposes. When used with highlyconfigurable manipulators, these processors may have joint controllerprogramming instructions or code which allows them to derive generallysuitable joint commands that could be used for one control task, such asmoving the end effector to a desired position. However, as themanipulators may have more degrees of freedom than are needed toaccomplish this task, the solution generated by the joint controllerwill often be underconstrained. In other words, one joint controller ofthe processor will often calculate a range of joint positions and/orrelationships, any of which could result in the desired end effectorposition in space.

Mathematically, the controller may perform at least some of thecalculations of the joint commands using vectors and/or matrices, someof which may have elements corresponding to configurations or velocitiesof the joints. The range of alternative joint configurations availableto the processor may be conceptualized as a joint space. The joint spacemay, for example, have as many dimensions as the manipulator has degreesof freedom, and a particular configuration of the manipulator mayrepresent a particular point in the joint space, with each coordinatecorresponding to a joint state of an associated joint of themanipulator.

When the solution calculated by a joint controller of the processor isunderconstrained, the solution provided by that joint controller mayrepresent a subset of the joint space. To identify what specificcommands are to be transmitted to the joints of the manipulator when aprimary solution is underconstrained, additional programminginstructions or code of the processor may effectively act as a subspacefilter, selecting a desirable manipulator state and specific set ofjoint commands from the range of alternatives generated by the jointcontroller. Advantageously, the selected commands can be used to serve asecond goal, task, or function. For example, when a primary jointcontroller is implemented as a velocity controller, such a filter canidentify a linear combination of joint velocities that are orthogonal tothe function of the primary joint controller, with these additionalvelocities driving the manipulator so that it passes through anaperture, toward a desired high-dexterity pose, and/or to inhibitcollisions. The filter will often be configuration-dependent, so thatthe joint commands selected by the filter will depend on theconfiguration or state of the joints, manipulator, and/or workspace. Infact, the primary joint controller may also effectively comprise afilter selecting the primary solution from the overall joint space basedon an input command and/or the like.

As used herein the term “overconstrained” encompass robotic systems inwhich a task or tasks of one or more controller will, at least at timesand if maintained as rigid constraints, be capable of exceeding theavailable degrees of freedom of an associated manipulator assembly.

As used herein, an “external” force or articulation of a robotic systemincludes forces of movement of the manipulator assembly which areapplied by a system user or other person, a workspace or environment, anunintended collision with another structure, and the like; but generallydoes not encompass robotically calculated and intended forces andmovements applied by driving the manipulator of the system.

As used herein, the term “null space” is defined as the subspace S of avector input space for a linear operator (a matrix) M such that for anyvector x in S, M·x=0.

While the processor is often described herein as having a primary jointcontroller and/or a configuration-dependent filter, and/or as having afirst module and a second module performing functions related to such acontroller and filter, the processors described herein may also have aplurality of filters (optionally being three or more filters), three ormore modules for three or more different control tasks, and the like.The processor will often define one or more priority between the tasksassociated with these filters and modules, thereby assigning greaterweight or importance to a higher priority task than a lower prioritytask. In many embodiments, even when a primary solution isunderconstrained, the tasks associated with the joint controller andfilter(s) may combine to overconstrain the system if such prioritieswere absent, so that the lower-priority tasks or goals may have littleor no effect on at least some manipulator movements.

Referring to FIG. 1A of the drawings, an operator workstation orsurgeon's console of a minimally invasive telesurgical system isgenerally indicated by reference numeral 200. The workstation 200includes a viewer 202 where an image of a surgical site is displayed inuse. A support 204 is provided on which an operator, typically asurgeon, can rest his or her forearms while gripping two master controls220 (see FIG. 2), one in each hand. The master controls or input devicesare positioned in a space 206 inwardly beyond the support 204. Whenusing workstation 200, the surgeon typically sits in a chair in front ofthe workstation, positions his or her eyes in front of viewer 202 andgrips the master controls, one in each hand, while resting his or herforearms on support 204. A processor 210 of the workstation generatessignals in response to the motion of the input devices.

In FIG. 1B of the drawings, a surgical station is generally indicated byreference numeral 300. In use, a patient P is supported by a table Tadjacent one or more manipulator support bases 302. Base 302 isgenerally supported from above, and may be ceiling mounted, supported bya wall of a room in which surgical station 300 is disposed, mounted to asurgical table, mounted to an arm of a cart having wheels or casters formoving the manipulators within the operating room or between operatingrooms, or the like. The structure supporting base 302 is not shown. Base302 will typically remain in a fixed location over patient P during atleast a portion of a surgical procedure. The workstation 200 (see FIG.1A) is typically positioned at some distance from the surgical station300, optionally being separated by a few feet within an operating room.In other embodiments, surgical station 300 and workstation 200 may beseparated by a significant distance, optionally being disposed inseparate rooms or even different buildings.

Surgical station 300 typically includes a plurality of roboticmanipulators 304, often having three or more robotic manipulators, withthe exemplary embodiment including four robotic manipulators supportedby base 302. Exemplary base 302 comprises an elongate base bodysupported in a horizontal orientation, with manipulators 304 beingdistributed horizontally along the length of the base. In otherembodiments, a plurality of separately positionable bases may supportthe manipulators.

As can be seen in FIGS. 1B and 1C, each of the robotic manipulatorssupports an associated surgical instrument 306. One or more of theinstruments may comprise an image capturing device 308 such as anendoscope or the like. Each of the other three manipulators 304 maysupport an instrument adapted for manipulating tissues at an internalsurgical site 310. Endoscope 308 is operatively connected to viewer 202to display an image captured at its viewing end on the viewer. Two ofthe other robotic manipulators 304 may each be operatively connected toone of the master controls, and a processor 210 may alter whichmanipulator is operatively connected with which master control. Thus,the movement of all the manipulators may be controlled by manipulationof the master controls. In some embodiments, additional input devicesmay be provided for use by another surgeon, a surgical assistant, or thelike.

An exemplary input device 220 and surgical instrument 306 areillustrated in FIGS. 2 and 3, respectively. Input device 220 includes anarm 222 and a wrist 224 which together allow translational andorientational movement of an input handle 226 relative to the structureof workstation 200 (see FIG. 1A). Handle 222 will generally move with aplurality of degrees of freedom relative to the workstation structure,the exemplary input device 220 providing six degrees of freedom ofmovement of handle 226. The linkage supporting the handle may includemore or less than six degrees of freedom. Grip members are movablycoupled to handle 226 and the handle generates a grip signal indicatingseparation between the grip members.

Referring now to FIG. 3, surgical tool or instrument 306 generallyincludes a surgical end effector 50 supported relative to a housing 53by an intermediate portion of the instrument, the intermediate portionoften comprising of an elongate shaft 14.1. End effector 50 may besupported relative to the shaft by a distal joint or wrist so as tofacilitate orienting the end effector within an internal surgicalworkspace. Proximal housing 53 will typically include an interface 232adapted for coupling to a holder of a manipulator 304. As described inmore detail in U.S. Pat. No. 6,331,181, the full disclosure of which isincorporated herein by reference, instrument 306 will often include amemory 230, with the memory typically being electrically coupled to adata interface (the data interface typically forming a portion ofinterface 232). This allows data communication between memory 230 andthe robotic surgical processor 210 of workstation 200 (see FIG. 1A) whenthe instrument is mounted on the manipulator.

Referring now to FIGS. 3A-3F, a variety of alternative robotic surgicalinstruments of different types and differing end effectors 50 may beused, with the instruments of at least some of the manipulators beingremoved and replaced during a surgical procedure. Several of these endeffectors, including DeBakey Forceps 56 i, microforceps 56 ii, Pottsscissors 56 iii, and clip a plier 56 iv include first and second endeffector elements 56 a, 56 b which pivot relative to each other so as todefine a pair of end effector jaws. Other end effectors, includingscalpel 56 v and electrocautery probe 56 vi have a single end effectorelement. For instruments having end effector jaws, the jaws will oftenbe actuated by squeezing the grip members of handle 226. Single endeffector instruments may also be actuated by gripping of the gripmembers, for example, so as to energize an electrocautery probe.

The elongate shafts 14.1 of instruments 306 allow the end effectors 50and the distal end of the shaft to be inserted distally into a surgicalworksite through a minimally invasive aperture, often through anabdominal wall or the like. The surgical worksite may be insufflated,and movement of the end effectors within the patient will often beeffected, at least in part, by pivoting of the instruments 306 about thelocation at which the shaft 14.1 passes through the minimally invasiveaperture. In other words, manipulators 304 will move the proximalhousing 53 outside the patient so that shaft 14.1 extends through aminimally invasive aperture location so as to help provide a desiredmovement of end effector 50. Hence, manipulators 304 will often undergosignificant movement outside patient P during a surgical procedure.

An exemplary structure of manipulator 304 can be understood withreference to FIGS. 4A-4C, 5A and 5B, and FIG. 6. As described above,manipulator 304 generally supports an instrument 306 and effectsmovements of the instrument relative to a base 302. As a number ofdifferent instruments having differing end effectors may be sequentiallymounted on each manipulator 304 during a surgical procedure (typicallywith the help of a surgical assistant), an instrument holder 320 willpreferably allow rapid removal and replacement of the mountedinstrument. The manipulator will often be covered by a sterile drapewhile the instrument may not be. An exemplary sterile adaptor and drapeare described in U.S. Pat. No. 6,331,181. As can be understood withreference the FIGS. 1B and 1C, manipulators 304 are mounted to base 302by a pivotal mounting joint 322 so as to allow the remainder ofmanipulator 304 to rotate about a first joint axis J1, with the firstjoint 322 providing rotation about a vertical axis in the exemplaryembodiment. Base 302 and first joint 322 generally comprise a proximalportion of manipulator 304, with the manipulator extending distally fromthe base toward instrument holder 320 and end effector 50.

Describing the individual links of manipulator linkage 304 asillustrated in FIGS. 4A-C along with the axes of rotation of the jointsconnecting the links as illustrated in FIG. 6, a first link 324 extendsdistally from base 302 and rotates about first pivotal joint axis J1 atjoint 322. Many of the remainder of the joints can be identified bytheir associated rotational axes in FIG. 6. For example, a distal end offirst link 324 is coupled to a proximal end of a second link 326 at ajoint providing a horizontal pivotal axis J2. A proximal end of a thirdlink 328 is coupled to the distal end of the second link 326 at a rolljoint so that the third link generally rotates or rolls at joint J3about an axis extending along (and ideally aligned with) axes of boththe second and third links. Proceeding distally, after another pivotaljoint J4, the distal end of a fourth link 330 is coupled to instrumentholder 320 by a pair of pivotal joints J5, J6 that together define aninstrument holder wrist 332. A translational or prismatic joint J7 ofthe manipulator facilitates axial movement of instrument 306 through theminimally invasive aperture, and also facilitates attachment of theinstrument holder to a cannula through which the instrument is slidablyinserted.

Distally of instrument holder 320, instrument 306 may include additionaldegrees of freedom. Actuation of the degrees of freedom of theinstrument will often be driven by motors of the manipulator, andalternative embodiments may separate the instrument from the supportingmanipulator structure at a quickly detachable instrumentholder/instrument interface so that one or more joints shown here asbeing on the instrument are instead on the interface, or vice versa. Inother words, the interface between the instrument and manipulator may bedisposed more proximally or distally along the kinematic chain of themanipulator assembly (which may include both the instrument andmanipulator). In the exemplary embodiment, instrument 306 includes arotational joint J8 proximally of the pivot point PP, which generally isdisposed at the site of a minimally invasive aperture. A distal wrist ofthe instrument allows pivotal motion of end effector 50 about instrumentwrist joint axes J9, J10. An angle α between end effector jaw elementsmay be controlled independently of the end effector location andorientation.

Referring now to FIGS. 7A-7C, a manipulator assembly 502 here includes amanipulator 504 and an instrument 506 having an end effector 508. Theterm manipulator assembly, as used herein, may in some cases alsoencompass the manipulator without the instrument mounted thereon. Theillustrated manipulator assembly generally extends from a proximal base510 distally to the end effector 508, with the end effector and distalportion of the instrument being configured for insertion into aninternal surgical site 512 via a minimally invasive surgical access 514.The joint structure of manipulator assembly 502 is similar to thatdescribed above regarding FIG. 6, and includes sufficient degrees offreedom so as to allow the manipulator assembly to be anywhere within arange of differing joint states for a given end effector position, evenwhen the instrument is constrained to passage through minimally invasiveaperture 514.

One of the challenges of working with the highly configurablemanipulator assembly of FIG. 7A can be understood with reference toFIGS. 7A and 7B. Specifically, when the access site to a minimallyinvasive surgical procedure is to be changed from a first aperturelocation 514 a to a second aperture location 514 b, it will often bedesirable to manually reposition some or all of the links of manipulator502. Similarly, when initially setting up the manipulator assembly forsurgery, the manipulator may be manually moved into a desired positionaligned with the aperture location through which the associatedinstrument is to access the surgical site. However, in light of thehighly configurable manipulator structure having a relatively largenumber of joints between (for example) base 510 and theinstrument/manipulator interface (see FIG. 6), such manual positioningof the links can be challenging. Even when the manipulator structure isbalanced so as to avoid gravitational effects, attempting to align eachof the joints in an appropriate arrangement can be difficult for oneperson, time consuming, and may involve significant training and/orskill. The challenges can be even greater when the links of themanipulator are not balanced about the joints, as positioning such ahighly configurable structures in an appropriate configuration to beginsurgery can be a struggle due to the manipulator's arm length and itspassive and limp design.

So as to facilitate setting up of the manipulator assembly 502 and/ormanipulator 504 for a surgical procedure (or reconfiguring themanipulator assembly for accessing a different tissue of the patient),the controller 210 of workstation 200 (see FIG. 1A) may actively drivejoints of the manipulator assembly during (and preferably in responseto) manual movement of at least one joint of the manipulator. In FIG.7C, a hand H of a system operator (optionally a surgeon, assistant,technician, or the like) manually moves a link of manipulator 504 orinstrument 506 into alignment with a desired minimally invasive aperture514 b. During this movement, the processor drives joints proximal of thehand/manipulator engagement. As the manipulator will often havesufficient degrees of freedom so as to be in a range of alternativeconfigurations, the proximal joints may be driven to a desiredmanipulator state without inhibiting the manual positioning of thedistal portion of the manipulator assembly. Optionally, the joints maybe driven so as to compensate for gravity, to inhibit momentum effects,to provide a desired (and often readily overcome) resistance to themanual movement so as to give the hand the impression of plasticallydeforming the manipulator structure at its joints, so as to keep theconfigurable linkage assembly in a desired pose, or the like. While thismovement is shown in FIG. 7C as being performed with instrument 506attached to manipulator 504, the manipulator assembly will often bemanually positioned prior to attachment of instrument 506 to themanipulator.

Referring now to FIGS. 7A and 7C, moving a location of a minimallyinvasive access site relative to base 510 may significantly alter adesired pose or configuration of the manipulator so as to maintain adesirable range of motion, avoid singularities of the manipulatorstructure, and the like. Taking advantage of the large number of degreesof freedom of the manipulator assembly, the processor may reconfigurethe joint states in a large variety of ways in response to the manualmovement of a link and/or articulation of one or more of the manipulatorassembly joints. The processor will often drive joints other than thatbeing articulated manually, optionally in combination with driving ofthe manually articulated joint so as to allow the link to be moved. Insome embodiments, movement of a minimally invasive access site mayresult from movement of the patient, optionally via movement of a tableon which the patient is supported, due to physiological movement such asbreathing or the like, and may optionally occur during manipulation oftissues at the internal surgical site by the end effector.

Referring now to FIG. 8E, another alternative manipulator assembly 520includes a manipulator linkage arm 522 for removably supporting asurgical instrument 524. A port clutch input 516 comprises an inputbutton which can be actuated by a hand engaging a link 518 of themanipulator that is to be disposed adjacent to access site 514 duringsurgery, such as the link to which the instrument holder is attached.This allows the hand to both actuate the input and help maneuver themanipulator into the appropriate configuration for surgery. In manyembodiments, the link 518 on which port clutch input 516 is disposedwill be coupleable to the shaft of the instrument by an axial insertionjoint (although the instrument may not be attached at the time). In someembodiments, the hand which actuates the port clutch input 516 may becapable of repositioning the manipulator in the clutch mode withoutassistance from another hand. In other embodiments, repositioning of themanipulator may be facilitated by having a user position both a firsthand on the manipulator link 518 adjacent port clutch input 516, and asecond hand at a distance from the port clutch input, particularly whenreorienting the link to a desired axial insertion angle. While portclutch input 516 is actuated by the hand, the system processor willdrive joints of manipulator 520 in response to manual movement of link518. In so doing, the processor will generally provide combinations ofjoint commands and/or velocities that limit the movement of themanipulator while it is in the clutch mode to one or more effectivedegrees of freedom. Surprisingly, coupling the joint movements together(for example, by filtering or allowing only desired linear combinationsof a manually articulated joint velocity with one or more roboticallydriven joint velocities) can provide one or more effective clutchdegrees of freedom that differ from each of the degrees of freedom ofthe individual joints. In most cases, more than one joint will beexternally articulated, and more than one associated joint velocitycommand will be calculated.

Three exemplary clutch modes (and some or all of their associatedeffective degrees of freedom) can be understood by reference to FIGS.8A-8D. In this embodiment of manipulator 520, three clutch inputs areshown, a port clutch input 516 a and an instrument clutch input 516 bmounted to link 518 adjacent the axial insertion axis of the manipulatorassembly, and a pose clutch input 516 c mounted along a link of themanipulator assembly between (and kinematically separated from) the base510 and the instrument holder. Other clutch modes and combinations arealso possible. The term “clutch” as used herein encompasses signaland/or data processing techniques which allow, for example, a roboticmanipulator assembly to be articulated manually, often while theprocessor of the robotic system provides joint command signals to atleast some of the joints of the manipulator assembly, and/or while aninput to a master/slave controller of the processor is temporarilyoperationally disassociated from the manipulator assembly, eitherpartially or fully. The terms “port clutch,” “instrument clutch,” and“pose clutch” broadly encompass clutching that involves or relates to anaperture (such as a minimally invasive aperture or port, or any otheropening in any structure), a robotic instrument or robotic tool, and apose or configuration of the manipulator, respectively.

As can be understood by comparing FIGS. 8A and 8B, and as generallydescribed above, actuation of port clutch input 516 a allows manipulator520 assembly to be reconfigured manually, generally allowing a systemuser to manually articulate at least one joint (and typically aplurality of joints) of the manipulator assembly. In response to themanual articulation, the processor may drive one or more joints of themanipulator assembly by transmitting signals to effect desiredcombinations of joint velocities of the joints, with the desiredcombinations varying with changes in the manipulator assemblyconfiguration. Hence, while at least one joint of the manipulatorassembly is being externally articulated (typically being articulated bya user or the like, rather than the robotic controller and drive system)at least one joint of the manipulator assembly is being roboticallydriven. The processor can thereby maintain a desired relationshipbetween the joint states so that at least a portion of the manipulatormoves with a desired constrained movement.

In FIG. 8B, for example, the processor constrains manual movement of aninstrument holder so as to translate an initial instrument shaft axis511 a to desired instrument shaft axis 511 b while maintaining anorientation of the axis throughout the movement. Where, for example, themanipulator assembly is moved from alignment with first aperture site514 a to alignment with a second aperture site 514 b at least in part bymanual articulation of joint J2, the processor calculates coordinatedmovements of joints J5, J7, and the like, in response to the changingjoint state of joint J2. The processor in the clutch mode calculatescommands for joints J5, J7, etc. so as to provide only the desiredtranslational movement, so that the manipulator assembly has aneffective translational degree of freedom which maintains theorientation of the instrument shaft axis. The shaft axis may betranslated in one, two or three degrees of freedom, and the instrumentneed not be mounted to the manipulator at the time the manipulator isbeing moved, as the instrument holder of link 518 can define theinstrument axis. A variety of linkages could provide similarmechanically constrained parallel-axis motion, but in this embodiment nomechanical joint(s) of the manipulator need provide such aparallel-motion degree of freedom.

Port and other clutch modes might provide a variety of alternativeclutch degrees of freedom, such as maintaining a desired pose of themanipulator assembly with ranges of motions maximized for themanipulator assembly configuration throughout translational and/orrotational movement, and the like. A variety of alternative coordinatedmanual and driven joint movements may be employed, with one, some, orall of the joints of the manipulator assembly being driven roboticallyin response to manual articulation of one, some, or all of the joints.External joint articulation may be sensed or monitored using joint statesensors of the robotic system.

Additional aspects of a port clutch mode may be provided, either insteadof or in combination with those described above. For example, once themanipulator assembly is aligned with the desired aperture site,articulation of port clutch input 516 a (such as by releasing the inputbutton) can prompt the processor to identify a location of the port.Link 518 (which supports the port clutch input 516 a) will often becoupleable to the instrument shaft by an axial joint, and a distal endof link 518 (and/or a cannula affixed thereto) will often be positionedat or adjacent to desired aperture site 514 b. The processor candetermine the location of the distal end of link 518 and or cannula fromthe joint states of the manipulator in response to actuation of the portclutch input 516 a, and can thereafter use this information to calculatemotions of the instrument shaft so that it pivots within desiredaperture site 514 b. Hence, the manipulator assembly in the port clutchmode can be used as an input for the aperture site location.

An exemplary instrument clutch mode can be understood with reference toFIG. 8C. Instrument clutching is useful after the manipulator assemblyis aligned with the aperture site, and can facilitate manually orientingof an instrument, cannula, or other structure of the manipulatorassembly toward an internal target tissue site, optionally after thecannula and/or end effector has been at least partially inserted intothe patient. In response to actuation of instrument clutch input 516 b,the processor drives the manipulator assembly so as to inhibit lateralmovement of shaft axis 511 at a pivotal center 513 adjacent aperturesite 514, while allowing lateral movement of the proximal end of link518. For example, in response to manual articulation of joints J6 andJ7, the processor may drive a combination of the manipulator joints toas to effect two-dimensional pivotal movements of link 518 centered atpivotal center 513, as schematically illustrated by arrows 515 and cone517. In some embodiments, movement of link 518 and/or the instrumentalong the shaft axis may be allowed, and in other embodiments such axialmovement of the instrument and/or link 518 may be inhibited. Regardless,a remote spherical center mechanical linkage may be simulated in theinstrument clutch mode without limiting the movements of the manipulatorassembly joints in other modes.

Yet another clutch mode, here a pose clutch mode, can be understood withreference to FIGS. 8A and 8D. In response to actuation of a pose clutchinput 516 c mounted to link 330 proximal of link 518, at least one jointof manipulator 520 can be articulated manually. In response, theprocessor drives one or more joints of the manipulator assembly so thatlink 518 (and its instrument holder) remains at a fixed position. Asmanipulator assembly 520 has a range of configurations for a givenposition of link 518 (and/or the instrument shaft, end effector, and thelike), the processor can coordinate motions of the joints so as to keepthe manipulator assembly within that range of configurations. Such apose clutch mode could allow a system user to manually reconfiguremanipulator assembly 518 from an apex downward configuration (as shownin FIG. 8A) to an apex upward configuration (as illustrated in FIG. 8D)while the processor maintains a position of a distal portion of themanipulator assembly.

Pose clutch input 516 c can provide a relatively simple approach toallow manual adjustment of the pose. The system processor will oftenmaintain the port and/or end-effector constraints when the manipulatorassembly is in a pose clutch mode, but may not apply any poseconstraint. Backdriveable manipulator assemblies may benefit from agravity compensation system (hardware and/or software) while in poseclutch mode, and such a pose clutch mode may allow manual adjustment ofthe pose without sensing forces (for example) between the base and thecannula. In pose clutch mode, the null space could be velocitycontrolled, and may not be position controlled.

While exemplary clutch inputs 516 a, 516 b, and/or 516 c may provideadvantages in simplicity of implementation, ease of understanding of thevarious clutch modes, and the like, a variety of other clutch mode userinterfaces might also be employed. For example, manipulator assemblieshaving systems capable of indicating forces applied to a cannula, an endeffector, or the like, a buttonless haptic clutch user interface mightbe implemented. Rather than pushing a button to alter a mode of theprocessor, the user might manually articulate the manipulator assemblyby applying haptic threshold-exceeding forces against appropriatestructures of the manipulator assembly. The processor may counteractforces below an appropriate haptic threshold, but may treat externalarticulations exceeding the threshold as an input into the manipulatorassembly, for example, by prioritized saturation of the end-effector,port, and/or pose constraints so that the system conforms to theexternal articulation.

Note that the saturation will generally not involve exceeding anyhardware capabilities of the manipulator assembly, but may instead beimplemented in software. For example, a processor can be configured todetermine that a predetermined force threshold has been exceeded in alateral direction on the cannula so as to indicate that complying with achange in the port location (and hence the pivotal center for future endeffector movements) is appropriate. The port position could bemaintained unless the forces applied at the cannula exceeds thethreshold. The external articulation of the manipulator assembly mightbe induced by a system operator intentionally trying to change the portposition, or by the patient's body wall such as when the patient movesor is repositioned on the table. In either case, if externalarticulation forces against the manipulator exceed the threshold value,the processor could allow the port to be moved to a new position,optionally while constraining manipulator movement to a desiredport-clutch mode as described above. When forces dropped below thethreshold, the processor could maintain the new port position.

A similar haptic force threshold could be applied to an instrumentclutch mode, particularly given a force sensing system capable ofsensing joint torques. In fact, a haptic force threshold instrumentclutch could be implemented on a mechanically constrained remote pivotalcenter robotic system. In either case, backdriving of the manipulatorusing external forces exceeding the threshold would cause the processorto modify the set point of the end effector so as to maintain the newposition, rather than returning the instrument to the prior position.Such instrument clutching may be most useful if implemented with a forcesensing system capable of distinguishing between forces applied withinthe body and forces applied outside the body. Instrument clutching mightthen only occur if forces applied to the manipulator outside the bodyexceeded the threshold.

The use of haptic force thresholds or haptic wells for alteringprocessor mode may be combined with processors configured to (typicallyusing software code) prioritize controller tasks such as maintainingport position, end-effector position, and pose, optionally in that inpriority order. Such force thresholds (and the other processor controltasks described herein) may be included in systems that implement thecontroller priorities using prioritized actuator torque saturation.Force thresholds may also be embodied using processors which drivemanipulators according to a primary solution and a secondary controltask having a priority therebetween, as described herein. Optionally,the primary priority may comprise holding the port position, and thesecondary priority may comprise maintaining end-effector position. Atertiary priority may comprise maintaining pose.

Referring now to FIGS. 9A and 9B, manipulator assembly 502 may bereconfigured by the processor for any of a variety of differing reasons.For example, a joint 526 may be driven from a downward oriented apexconfiguration to an upward oriented apex configuration so as to inhibitcollisions with an adjacent arm, equipment, or personnel; to enhance arange of motion of the end effector 508; in response to physiologicalmovement of the patient such as patient breathing or the like; inresponse to repositioning of the patient, such as by reorienting asurgical table; and the like. Some, but not all, of these changes inconfiguration of the manipulator assembly may be in response to externalforces applied to the manipulator assembly, with the processor oftendriving a different joint of the manipulator than that which is beingacted upon by the external force. In other cases, the processor willreconfigure the manipulator assembly in response to calculationsperformed by the processor. In either case, the processor may vary froma simple master-slave controller so as to drive manipulator assembly inresponse to a signal so as to provide a preferred manipulator assemblyconfiguration. Such configuring of the manipulator assembly may occurduring master-slave end effector movements, during manual or otherreconfiguration of the manipulator assembly, and/or at least in part ata different time, such as after releasing a clutch input.

Referring now to FIG. 10A, a simplified controller schematic diagram 530shows a master/slave controller 532 coupling a master input device 534to a slave manipulator 536. In this and many of the other controllersdescribed herein, the controller inputs, outputs, and computations aredescribed using vector mathematical notation in which the vector x willoften refer to a position vector in a Cartesian coordinates, and inwhich the vector q will reference a joint articulation configurationvector of an associated linkage (most often of the manipulator slavelinkage), sometimes referred to as the linkage position in joint space.Subscripts can be appended to these vectors to identify a specificstructure when ambiguity might otherwise exist, so that x_(m) (forexample) is a position of the master input device in the associatedmaster workspace or coordinate system, while x_(s) indicates a positionof the slave in the workspace. Velocity vectors associated with theposition vectors are indicated by a dot over the vector or the word“dot” between the vector and the subscript, such as xdot_(m) or {dotover (m)}_(m) for the master velocity vector, with the velocity vectorsbeing mathematically defined as the change in the position vector with achange in time (dx_(m)/dt for the master velocity vector example).

In the embodiment of FIG. 10A, controller 532 comprises an inverseJacobian velocity controller. Where x_(m) is a position of the masterinput device and {dot over (x)}_(m) is the velocity of the master inputdevice, the controller 532 calculates motor commands for transmission tothe manipulator 536 so as to effect slave end effector motions whichcorrespond to the input device from the master velocities. Similarly,controller 532 can calculate force reflection signals to be applied tothe master input device (and from there to the operator's hand) from theslave position x_(s) and/or slave velocity {dot over (x)}_(s). A numberof refinements to this simple master/slave inverse Jacobian controllerschematic are desirable, including those illustrated in FIG. 11 anddescribed in detail in U.S. Pat. No. 6,424,885, the full disclosure ofwhich is incorporated herein by reference.

Referring now to FIG. 10B, a processor 542 may be characterized asincluding a first controller module 544 and a second controller module546. The first module 544 may comprise a primary joint controller, suchas an inverse Jacobian master-slave controller. The primary jointcontroller of first module 544 may be configured for generating thedesired manipulator assembly movements in response to inputs from themaster input device 534. However, as noted above, many of themanipulator linkages described herein have a range of alternativeconfigurations for a given end effector position in space. As a result,a command for the end effector to assume a given position could resultin a wide variety of different joint movements and configurations, someof which may be much more desirable than others. Hence, the secondmodule 546 may be configured to help drive the manipulator assembly to adesired configuration, in some embodiments driving the manipulatortoward a preferred configuration during master-slave movements. In manyembodiments, second module 546 will comprise a configuration dependentfilter.

In broad mathematical terms, both the primary joint controller of firstmodule 544 and the configuration dependent filter of second module 546may comprise filters used by processor 542 to route control authorityfor linear combinations of joints to the service of one or more surgicalgoals or tasks. If we assume that X is the space of joint motion, F(X)might be a filter giving control over the joints to i) provide a desiredend effector movement, and ii) provide pivotal motion of the instrumentshaft at the aperture site. Hence, the primary joint controller of firstmodule 544 may comprise filter F(X). Conceptually, (1−F¹F)(X) coulddescribe a configuration dependent subspace filter giving controlactuation authority to the linear combination of joint velocities thatare orthogonal to serving the goal of the primary joint controller (inthis example, end effector movement and pivotal instrument shaftmotion). Hence, this configuration dependent filter could be used by thesecond module 546 of controller 542 to service a second goal, such asmaintaining a desired pose of the manipulator assembly, inhibitingcollisions, or the like. Both filters may be further sub-divided intomore filters corresponding to serving more specific tasks. For example,filter F(X) could be separated into F₁(X) and F₂(X) for control of theend effector and control of the pivotal shaft motion, respectively,either of which may be chosen as the primary or highest priority task ofthe processor.

While the mathematical calculations performed by the modules may (atleast in part) be similar, the robotic processors and control techniquesdescribed herein will often make use of a primary joint controllerconfigured for a first (sometimes referred to as a primary) controllertask, and a configuration dependent filter which makes use of anunderconstrained solution generated by the primary joint controller fora second (sometimes referred to as secondary) task. In much of thefollowing description, the primary joint controller will be describedwith reference to a first module, while the configuration dependentfilter will be described with reference to a second module. Additionalfunctions (such as additional subspace filters) and or additionalmodules of varying priorities may also be included.

As noted elsewhere herein, the hardware and/or programming code forperforming the functions described with reference to such first andsecond modules may be fully integrated, partially integrated, or fullyseparate. Controller 542 may employ the functions of the two modulessimultaneously, and/or may have a plurality of differing modes in whichone or both modules are used separately or in different ways. Forexample, in some embodiments, first module 544 might be used with littleor no influence from second module 546 during master-slavemanipulations, and the second module 546 having a greater role duringsetup of the system when the end effector is not being drivenrobotically, such as during port clutching or other manual articulationsof the manipulator assembly. Nonetheless, in many embodiments bothmodules may be active most of or all the time robotic motion is enabled.For example, by setting gains of the first module to zero, by settingx_(s) to x_(s, actual), and/or by reducing the matrix rank in theinverse Jacobian controller so that it doesn't control as much andletting the configuration dependent filter have more control authority,the influence of the first module on the state of the manipulatorassembly can be reduced or eliminated so as to change a mode ofprocessor 542 from a tissue manipulator mode to a clutch mode.

FIG. 10C illustrates a refinement of the simplified master-slave controlschematic 540 from FIG. 10B, and shows how different modules might beused in different processor modes. As illustrated in FIG. 10C, firstmodule 544 may, for example, comprise some form of a Jacobian controllerhaving a Jacobian-related matrix. Second module 546 may, in a portclutch mode, receive signals from the slave 536 indicating a position orvelocity of the slave generated at least in part by manual articulationof the slave manipulator linkage. In response to this input, the secondmodule 536 can generate motor commands appropriate for driving thejoints of the slave so as to allow the manual articulation of the slavelinkage while configuring the slave in the desired joint configuration.During master-slave end effector manipulation, the controller may usesecond module 546 to help derive motor commands based on a differentsignal b{dot over (q)}₀. This alternative input signal to the secondmodule 546 of controller 542 may be used to drive the manipulatorlinkage so as to maintain or move the minimally invasive aperture pivotlocation along the manipulator structure, so as to avoid collisionsbetween a plurality of manipulators, so as to enhance a range of motionof the manipulator structure and/or avoid singularities, so as toproduce a desired pose of the manipulator, or the like. Hence, b{dotover (q)}₀ can generally comprise and/or indicate (for example) adesired set of joint velocities, more generally representing a secondarycontrol goal, typically in joint space. In other embodiments, theprocessor may include separate modules and/or dependent configurationfilters for clutching, secondary controller tasks, and the like

Referring now to FIG. 12, a partial control schematic 550 illustratesmodifications of the controller illustrated in FIG. 11. Controlschematic 550 very roughly represents a modification of portion 551 ofthe controller of FIG. 11 to facilitate control over manipulatorassemblies have large numbers of degrees of freedom. In the embodimentillustrated in FIG. 12, the first module 544 comprises an inverseJacobian velocity controller, with the output from calculations madeusing an inverse Jacobian matrix modified according to a virtual slavepath 552. First describing the virtual slave path, vectors associatedwith the virtual slave are generally indicated by a v subscript, so thata virtual slave velocity in joint space {dot over (q)}_(v) is integratedto provide q_(v), which is processed using an inverse kinematic module554 to generate a virtual slave joint position signal x_(v). The virtualslave position and master input command x_(m) are combined and processedusing forward kinematics 556. The use of a virtual slave (often havingsimplified dynamics) facilitates smooth control and force reflectionwhen approaching hard limits of the system, when transgressing softlimits of the system, and the like, as can be more fully understood withreference to the '885 patent previously incorporated herein byreference. Similarly, calculation of motor commands such as joint torquesignals or the like from joint controllers in response to the outputfrom the inverse Jacobian matrix (as modified or augmented by the secondmodule 546) via appropriate joint controllers, input and outputprocessing, and the like are more fully described in the '885 patent.

Addressing the structure generally indicated by the first and secondcontrol modules 544, 546, and of the other components of controlschematic 550 and other controllers described herein, these structureswill often comprise data processing hardware, software, and/or firmware.Such structures will often include reprogrammable software, data, andthe like, which may be embodied in machine-readable code and stored in atangible medium for use by processor 210 of workstation 200 (see FIG.1A). The machine-readable code may be stored in a wide variety ofdifferent configurations, including random access memory, non-volatilememory, write-once memory, magnetic recording media, optical recordingmedia, and the like. Signals embodying the code and/or data associatedtherewith may be transmitted by a wide variety of communication links,including the Internet, an intranet, an Ethernet, wireless communicationnetworks and links, electrical signals and conductors, optical fibersand networks, and the like. Processor 210 may, as illustrated in FIG.1A, comprise one or more data processors of workstation 200, and/or mayinclude localized data processing circuits of one or more of themanipulators, the instruments, a separate and/or remote processingstructure or location, and the like, and the modules described hereinmay comprise (for example) a single common processor board, a pluralityof separate boards, or one or more of the modules may be separated ontoa plurality of boards, some of which also run some or all of thecalculation of another module. Similarly, the software code of themodules may be written as a single integrated software code, the modulesmay each be separated into individual subroutines, or parts of the codeof one module may be combined with some or all of the code of anothermodule. Hence, the data and processing structures may include any of awide variety of centralized or distributed data processing and/orprogramming architectures.

Addressing the output of the controller of FIG. 12 in more detail, thecontroller will often seek to solve for one particular manipulator jointconfiguration vector q for use in generating commands for these highlyconfigurable slave manipulator mechanisms. As noted above, themanipulator linkages often have sufficient degrees of freedom so as tooccupy a range of joint states for a given end effector state. Suchstructures may (but will often not) comprise linkages having trueredundant degrees of freedom, that is, structures in which actuation ofone joint may be directly replaced by a similar actuation of a differentjoint along the kinematic chain. Nonetheless, these structures aresometimes referred to as having excess, extra, or redundant degrees offreedom, with these terms (in the broad sense) generally encompassingkinematic chains in which (for example) intermediate links can movewithout changing the position (including both location and orientation)of an end effector.

When directing movement of highly configurable manipulators using thevelocity controller of FIG. 12, the primary joint controller of thefirst module often seeks to determine or solve for a virtual jointvelocity vector {dot over (q)}_(v) that can be used to drive the jointsof slave 536 in such a way that the end effector will accurately followthe master command x_(m). However, for slave mechanisms with redundantdegrees of freedom, an inverse Jacobian Matrix generally does not fullydefine a joint vector solution. For example, the mapping from Cartesiancommand {dot over (x)} to joint motion {dot over (q)} in a system thatcan occupy a range of joint states for a given end effector state is amapping of one-to-many. In other words, because the mechanism isredundant, there are a mathematically infinite number of solutions,represented by a subspace in which the inverse lives. The controller mayembody this relationship using a Jacobian matrix that has more columnsthan rows, mapping a plurality of joint velocities into comparativelyfew Cartesian velocities. Our solution J⁻¹{dot over (x)} will often seekto undo this collapsing of the degrees of freedom of the slave mechanisminto the Cartesian workspace.

The primary joint controller of the first module may employ a number ofdifferent techniques to generate such a primary solution.Mathematically, when a linear map (such as a Jacobian matrix) ismany-to-one, it may be said to have a non-trivial null space, the nullspace typically comprising a subspace of the input space spanning one ormore dimensions. For linear maps with a non-trivial null space, variouspseudo inverses may be constructed to pick an acceptable inversesolution vector falling within the null space. These pseudo inversestypically have zero residual error, and the pseudo inverse solutionequation can be written as

{dot over (q)}={dot over (q)} _(part) +{dot over (q)} _(h)

Here the pseudo inverse solution {dot over (q)} is equal to thecombination of any particular solution {dot over (q)}_(part) added tothe homogeneous solution {dot over (q)}_(h). Addressing the first termor particular solution, as the particular solution solves the inverseproblem, we know that

{dot over (x)}=J{dot over (q)} _(part)

The least-mean-squares solution, sometimes referred to as the minimumtwo-norm solution, is one exemplary choice for the first term of theabove general joint space solution equation, as it provides oneparticular solution to the Jacobian. This pseudo inverse yields thefinal solution vector with ∥{dot over (q)}_(part)+{dot over (q)}_(h)∥₂minimized, and is among the alternative pseudo inverses described below.The second or homogeneous solution term of the above pseudo inversesolution equation is a mapping into the null space of Jacobian matrix Jso that:

0=J{dot over (q)} _(h).

We can add any amount of {dot over (q)}_(h) to {dot over (q)}_(part) andnot affect at all the motion of the mechanism regarding the controlobjective, {dot over (x)}. Appropriate suitable pseudo inverses willtypically be designed using some sort of additional constraint toeffectively determine what portion (if any) of the null space to add toa particular solution so as to fully define the solution used tocalculate motor commands.

The least squares or minimum norm inverse will be indicated herein asJ^(#). In the case where Jacobian J is underdetermined, the minimum norminverse J^(#) can be identified from:

J ^(#) =J ^(T)(JJ ^(T))⁻¹

Where J^(T) is the transpose of the Jacobian matrix. This inverse isminimum-norm in the sense that the two-norm of the resulting solutionvector {dot over (q)}=J^(#) {dot over (x)} is generally the smallestnorm over the subspace of the solution vectors (remembering that thereare in an infinite subspace of such vectors). For a surgical robot (orfor any other robot), this generally corresponds to a solution thatachieves the primary control objective (such as a master-slave movementof the end effector corresponding to that of a handle of the inputdevice) with the smallest joint velocities, a reasonable goal for manyrobotic systems. It is also mathematically extremely convenient, beingone of the easiest solutions to compute. However, this is not the onlypseudo inverse of Jacobian J, and indeed there may be more appropriatetask-based solutions, particularly when there are additional constraintsor goals for movement of a system beyond (in our initial example) acommanded movement of the end effector with a minimum joint velocity. Asthe minimum norm inverse is just one example of a pseudo-inverse, whereJ⁻¹ or J^(#) appears herein it will often be reasonable to think ofthese as generally representing any pseudo-inverse, not merely theminimum norm inverse.

From the mechanisms described above, it will be understood that thedesired control space {dot over (x)} may optionally be more complex thansimple end effector motion, particularly in systems in which thecomplexity of redundant manipulator linkages is justified. Combining ofcontroller tasks or objectives will generally make use of aconfiguration dependent filter of the second module. A variety ofdifferent filters may be used. In some cases, multiple objectives may beadjoined or stacked to form an augmented desired vector, particularly ifthe objectives are independent. For example, in the surgical robotmanipulator assembly 502 illustrated in FIG. 7a , we can stack oraugment the commanded end effector vector with a desired cannula orpivot point location, so as to provide a software-induced center ofpivotal motion at a minimally invasive aperture site. Such pivotalmotion is sometimes referred to as a software center motion. Augmentingthe problem statement in this manner is most easily applicable when themanipulator assembly has sufficient degrees of freedom to provide asolution to the augmented equation, as there generally are in theabove-described robot manipulator assembly linkages. That is, theaugmented Cartesian command vector (with the pivotal motion constraint){dot over (q)} still has a lower dimension than the joint velocityvector {dot over (q)}, whose dimension is typically the number ofmanipulator joints.

An exemplary augmented end effector and cannula control problemformulation may comprise computing a Jacobian J for each controlobjective. For controlling the end effector, we have J_(eff) mapping{dot over (q)} to {dot over (x)}_(eff) where _(eff) is here used toindicate the vector and/or matrix that is associated with the endeffector. For controlling the cannula, we also construct a JacobianJ_(can) that maps the joint velocities of the manipulator assembly tothe cannula Cartesian velocities, written {dot over (x)}_(can). Thus themapping can be written:

$\begin{bmatrix}{\overset{.}{X}}_{eff} \\{\overset{.}{X}}_{can}\end{bmatrix} = {\begin{bmatrix}J_{eff} \\J_{can}\end{bmatrix}\overset{.}{q}}$

By using the above mapping, the pseudo inverse can provide aminimum-norm solution to control both the end effector and the cannulavelocities. As indicated in the control diagram of FIG. 12, positionloops may be closed using signals from the manipulator assembly slave536 to control positions (including location and orientation) as well.

Additional control objectives, such as driving toward a preferredcollision-inhibiting pose or the like, may also be added in a similarmanner. However, as such objectives are added the dimension of thedesired vector space {dot over (x)} may equal or surpass that of thesolution space {dot over (q)}. Here, the subspace filter of the secondmodule may take another form.

When the dimensions of the vector space and solution space are equal,the pseudo inverse should become a true inverse, and the minimum-normpseudo inverse conveniently reduces to the true inverse. Should moreaugmented solutions be sought, however, the controller and the robotwill not in general have sufficient degrees of freedom to satisfy allthe constraints. Such a problem is described as over-constrained, sothat a new kind of pseudo inverse may be created. Such a pseudo inverseattempts to compromise in some mathematical sense, returning a solutionthat does not perfectly satisfy all of the constraints but may still bemathematically determined to be the best solution in some identifiablesense. For example, a residual error resulting from the imperfectinverse may be computed, such as by using the minimum two-norm pseudoinverse to return a solution that minimizes the two-norm of the residualerror.

Examples of additional constraints that may be added include maintaininga pose constraint, and/or avoiding a collision with an object which maybe present in the workspace. In general, if a cost function C describingthese or other objectives can be written in a quadratic form, we cangenerally state that:

C=r ^(T) r=({dot over (x)} _(k)−Σ_(j-1) ^(n) J _(kj) {dot over (q)}_(j))².

Here {dot over (x)}_(k) is the desired control variable; {dot over(q)}_(j) is the joint velocities of joints 1, 2, . . . n; J_(kj) are theJacobians of the desired control variable for each joint, and r is theresidual error appropriate for our quadratic cost function C. Thepartial derivatives can generally be taken and set equal to zero,allowing us to solve for a two-norm or least squares solution:

$\frac{\partial C}{\partial{\overset{.}{q}}_{j}} = {0 = {\frac{\partial}{\partial{\overset{.}{q}}_{j}}\left( {{\overset{.}{x}}_{k}{\sum\limits_{j = 1}^{n}{J_{kj}{\overset{.}{q}}_{j}}}} \right)^{2}}}$

Hence, taking partial derivatives of a quadratic cost function with theappropriate Jacobian matrix can lead to the minimum norm solution,regardless of the number of constraints. Where the system isover-constrained, the resulting solution has a minimum cost function inwhich the norm of r is minimized. If the system is under-constrained,the resulting solution has zero cost function and solution takes theform of the minimum two-norm solution, so that {dot over(q)}_(optional)=J^(T)(JJ^(T))⁻¹{dot over (x)}.

As a simple example of an augmented controller, we can control a jointto maintain a pose using the above framework by creating a desiredcontrol variable {dot over (x)}_(k) and making it identical to aselected joint {dot over (q)}_(j). We then write an augmented controlleras described above, but here controlling the selected joint's desiredvelocity to be equal to zero. The augmented Jacobian can in this examplebe a trivial matrix with a one in the column for the selected joint. Wecould then close the loop around a desired joint angle, using the blockdiagram of FIG. 12 in a manner similar to that of FIG. 11 while drivingthe selected joint toward the desired joint angle. In other words, if anembodiment of the controller of FIG. 12 were used to drive a manipulatorassembly having one extra degree of freedom, but drove the extra degreeof freedom to a fixed configuration, the controller would give resultssimilar to those achieved by a fully constrained manipulator assemblydriven by the controller of FIG. 11, thereby indicating that thecontroller of FIG. 12 is capable of performing the primary function ofproducing a desired end effector movement, along with the additional (ifsomewhat simple) task of keeping one joint fixed.

The analysis is more complex for an over-constrained system incomparison to an under-constrained system. Robots in general, andsurgical robots in particular, should be designed and controlled so asto make advantageous use of all their joints. As the demands placed onthe kinematic system of a robotic manipulator vary, it is not alwayspractical or desirable to include sufficient degrees of freedom forthese robotic manipulators to simultaneously perform different tasks,even if it is occasionally desirable to perform these different tasks atthe same time. As a result, these systems can often be over-constrainedduring at least a portion of a robotic manipulation. There are severalmathematical frameworks through which constraints can be added usingdifferent subspace filters. The first is augmented desired vectors, asintroduced above. Some additional augmentation techniques which may beapplied are discussed below, after which alternative mathematicalframeworks, including at least one having advantages over augmentation,will also be described.

Along with simple combinations or augmentation of commands, a pluralityof commands may optionally be combined by the controller with associatedweights, for example by applying a weighted minimum norm solution. Theminimum norm can be computed with respect to a set of basis vectorsrepresenting the solution space. The solution may then comprise a linearcombination of those basis vectors. While basis vectors are sometimesvisualized as one unit in the x direction, one unit in the y direction,etc., this is not necessarily required. For example, joints may differsignificantly, with some being axial or prismatic, others being pivotalor rotational, and the like.

The use of basis vectors may be understood from a simple example, inwhich e₁ is one degree/sec for a first motor, e₂ is one degree/sec for asecond motor, etc., and in which an initial minimum norm solution spaceis computed. This solution space could be arbitrarily weighted byinstead choosing different basis vectors. For example, we could make e₁equate to ten degrees/sec for the first motor, while e₂ remains onedegree/sec for the second motor. A minimum norm solution will tend tomake coefficients for e₁ no bigger, on average, than those for e₂ andthe rest. Thus, this weighting approach can tend to make the firstmotor's velocity smaller, because of the weighting of basis functions wehave chosen.

Further, one may choose e₁ to be one degree/sec in the positivedirection for the first motor, and one degree/sec in the positivedirection for the second motor. e₂ could then be one degree/sec in thepositive direction for the first motor, and one degree/sec in thenegative direction for the second motor. Again, we can therebyeffectively change the weighting and/or the way the compromises aremapped into the space that the surgeon cares about. In general, we maywrap any transformation (it is helpful if the transformation is linear,constant, and invertible) around the optimization, as in:

C _(weighted) =r ^(T) W ^(T) Wr,

where W is a weighting matrix, and r is the residual error. In thissense, we weight the residual error as desired, thereby helping to shapethe answer.

Hence, weighting can be used to make some parts of the solution moreimportant than others. For example, W can be made diagonal with largeweights for important components and small weights for less importantcomponents of the augmented solution vector. Alternately, linearcombinations of any solution components can be arbitrarily weighted.

Another technique which might be applied to derive motor commands forover constrained systems is to make use of techniques for optimizationof noisy, ill-conditioned maps. All finite dimensional linear inverseproblems have a condition number. This number is a measure of the skewor lopsidedness of the linear map. Technically, it is a ratio of thelargest singular value of the linear map to the smallest, written

$\frac{\sigma_{\max}}{\sigma_{\min}}.$

Conceptually, it describes how noise and signal interact.

If a measurement is noisy (such as the measurement of velocity of arobotic joint or a master command) and the noise is independent acrossjoints, then it might be tempting to assume the would corrupt eachsolution component about equally. That is, if the desired velocity fromthe master, measured in the presence of noise, is {dot over(x)}_(k,noise)={dot over (x)}_(k,true)+GWN (μ,σ), (where μ and σ are themean and variance of a Gaussian white noise process GWN, and areconstant across the different Cartesian elements xdotk) then one mightexpect (albeit incorrectly) that {dot over (x)}_(j,noise)={dot over(q)}_(j,noise)+GWN({tilde over (μ)},{tilde over (σ)}) in which {tildeover (μ)},{tilde over (σ)} are also constant across the joints. In factthis is not so, and such an assumption would be in error—the additivenoise is shaped by the action of the linear map, which adds somecomponents together in a way that tends to build signal-to-noise ratio(“SNR”), and adds others in a way that tends to erode SNR. Considerationof the action of a poorly designed weighting matrix will clarify thatthis is possible. The measure of this effect is the condition number.

For robotic systems, the loss of precision associated with an increasein condition number happens when the Jacobian becomes ill conditioned,which in turn happens near singularities. As the condition number grows,the map acts more and more like its null space is growing by one or moredimension, but the effect is continuous, not discrete. Instead, a newsubspace of joint (solution) vectors map almost to zero in the desired(master) vector space. Any noise, even round-off error in numericcalculations, can and often does result in phantom solutions, in whichunrealistic joint velocities are commanded for minor velocities incommand space, including minor contributions due to noise.

A number of subspace filter techniques can address this ill-conditionedmap effect, including conditioning the solution by truncating thesingular value decomposition (“SVD”) of the Jacobian, effectivelyforcing the nearly grown null space to grow a full dimension. Thefollowing are brief descriptions of some of these techniques, includingthis SVD truncation.

One technique to address an ill-conditioned map, which may be motivatedby numerical approachability as much as anything else, is to reduce thecondition number in a numerically tractable manner. This techniquesolves a problem related to the original inverse linear problem, and canbe written as:

J*=J ^(T)(JJ ^(T)30 λ1)⁻¹

J* may be referred to as the regularized Jacobian, and I is the identitymatrix. The positive constant λ is the regularization constant. It hasthe effect of increasing all the singular values of the linear map by afixed amount, √{square root over (λ)}. This reduces the condition numberas a rising tide raises all boats: the original condition number

$\frac{\sigma_{\max}}{\sigma_{\min}}$

becomes the reduced condition number

$\frac{\sigma_{\max} + \sqrt{\lambda}}{\sigma_{\min} + \sqrt{\lambda}}.$

The disadvantage of the regularizing inverse is that the choice of λ isnot always well motivated physically. Further, it can change thesolution for all components of the inverse, not just the ones that areswamped by noise.

Another subspace filter approach to improve the condition of anill-conditioned map is Bayesian estimation, which is also referred to asmaximum a posteriori estimation. In its simplest form it reduces tosomething equivalent to a regularizing inverse. The main concept is tofind the free parameters {dot over (q)}_(j) that maximize the aposteriori probability density function of a noisy measurement p({dotover (q)}_(j), {dot over (x)}₁, . . . , {dot over (x)}_(n)). Via Bayes'rule, we may write

p({dot over (q)} _(j) ,{dot over (x)} ₁ , . . . ,{dot over (x)}_(n))p({dot over (x)} ₁ , . . . ,{dot over (x)} _(n))=p({dot over (x)} ₁, . . . ,{dot over (x)} _(n) ,{dot over (q)} _(j))p({dot over (q)} _(j))

so that the optimal joint velocity vector {dot over ({tilde over(q)})}_(j) is obtained by

$\overset{\sim}{\overset{.}{q_{j}}} = {\underset{{\overset{.}{q}}_{j}}{\arg \; \max}{p\left( {{\overset{.}{x}}_{1},\ldots \mspace{14mu},{\overset{.}{x}}_{n},{\overset{.}{q}}_{j}} \right)}{p\left( {\overset{.}{q}}_{j} \right)}}$

Here the first of the two new probability density functions (“PDFs”) isthe measurement noise process, and the second is the a priori (or prior)distribution for the solution vector. It can be shown that, if the noiseprocesses are zero mean, independent and Gaussian, and if the priorestimates on the solution are also zero mean, independent and Gaussian,then the solution may be generally written as:

{dot over ({tilde over (q)})}=J ^(T)(JJ ^(T) +AA ^(T))⁻¹ {dot over (x)}

where A is a matrix formed of the variances of the measurement processdivided by the variances of the prior distribution. As with aregularizing inverse, Bayesian methods change the solution. In thiscase, the change is physically motivated by an optimization techniquemeant to handle measurements in the presence of noise, and to handle theavailability of prior information about the probable solution.

As a final note regarding subspace filtering of ill-conditioned maps,similar truncation methods using the singular value decomposition can beused with great effect. In particular, this approach can be applied inthe control system illustrated in FIG. 11 with a six-by-six Jacobian.These methods are also extensible to the over- and under-constrainedcases. One disadvantage of this approach is that the overhead ofcomputing the inverse can be higher than other techniques discussedhere, although not necessarily by much. An advantage is that thedamaging condition number may be directly inspected and controlled.

A still further augmented optimization subspace filter approach whichmay be applied to over constrained redundant degree of freedom surgicalrobotic systems is constrained convex optimization. A minimum normsolution based on minimizing a quadratic form has the mathematicalelegance of reducing to a system of linear equations. In general, thesesimultaneous linear equations may be solved rapidly and with highaccuracy, and are approachable and easy to understand, and this ease ofimplementation certainly encourages their use. It is possible to writeother optimization problem statements, including some which may be wellposed and also achieve solutions almost as readily. A class of suchproblems is termed convex optimization, and a corresponding class ofalgorithms exists that yield efficient solutions, including thosealgorithms described by Boyd and Vandenberghe.

In addition to (or as an alternative from) the methods described above,controllers of surgical robotic systems having redundant degrees offreedom might also benefit from mathematical approaches for addressingconstraints that respect priority. The above discussion has largelycentered on optimization of a problem where constraints are imposedusing augmented optimization. However, there may be differences inimportance between a central, main control problem and any desirable,but not necessarily required solutions that we can add in. Severalmathematical mechanisms are introduced below for making constraintssoft, so that they fold back or can be relaxed gracefully as the maincontrol problem gets harder, employing more of the degrees of freedom ofthe system. An example of a soft constraint would be to maintain anelbow or joint apex of a manipulator pointing up, as described aboveregarding FIGS. 5A and 5B. Another constraint might be to maximize therobot's workspace dexterity, or some other mathematical measure orrobotic effectiveness. We might, for example, wish our controller togradually abandon these secondary goals as the primary task (forexample, complying with an end effector movement command vector),thereby allowing the system to achieve the primary objective at the costof these soft, lower priority goals. For example, we might need and usethe otherwise extra degrees of freedom to avoid a singularity or thelike.

One way to establish a system having a plurality of constraints, tasks,or goals with priorities therebetween is to construct the secondary taskas an augmented part of the solution, per the discussion above. We canthen use a weighting matrix to reduce the importance of the secondaryconstraint as compared to that of the primary task. We can also adjustthis weighting matrix during use so that the weighting is neutral whenthe robot is in its sweet spot, while the weighting might exert a verystrong clamp on any augmented solution (making the secondary constraintcontribute almost nothing to the final solution) as the difficult regionwas approached. It is also possible to turn on and off the weightingaltogether, so as to achieve a discrete switch that respects priority.Similar techniques may be used for the regularized solution and theBayesian approach. For example, the parameter 2, as well as the priordistribution, may be tuned in a configuration dependent manner.

An alternative mathematical approach which can allow a control system toimplement a plurality of constrains wherein the constraints have apriority therebetween is sometimes referred to as the projector method.These advantageous methods generally make use of matrix projection toadd constraints, and can be used instead of and/or in addition to theaugmentation methods described above.

Referring once again to the block diagram of FIG. 12, projector methodsmay make use of input vector b{dot over (q)}₀. We may first determinethat our primary constraint or task is that the master-slave movement ofthe end effector generated by the manipulator assembly should, aspracticable, always correspond to the commanded movement of the masterinput device. For any linear map of this main constraint or optimizationof the end effector movement, J_(eff) (mapping {dot over (q)} to {dotover (x)} for the end effector) we can construct a filter for one ormore other secondary constraint or command vector {dot over (q)}₀ sothat this secondary constraint(s) cannot interfere with the primarycommanded motion {dot over (x)}_(m). This avoidance of interferencebetween the constraints can be achieved by constructing or limiting {dotover (q)}₀ so that it is entirely within the null space of J_(eff) Sucha limit or filter works as a mathematical projector, projecting ageneral constraint or goal {dot over (q)} into the special, prioritizedgoal {dot over (q)}₀ with the property J_(eff){dot over (q)}₀=0. Thereare many such projectors, exactly one of which is lossless. Appropriateprojectors P include the lossless projector:

P=1−J _(eff) ⁻¹ J _(eff)

where J⁻¹ _(eff) is any pseudoinverse appropriate for use.

Now consider any task that can be written as a linear controller ofrates with a position feedback, such as the remote center cannula motiontask described above. The Jacobian can be formed as before, and thejoint velocities responsible for achieving the task may be computed (forexample) as:

{dot over (q)} _(can) =J _(can) ^(#) {dot over (x)} _(can).

The proposed rates for the cannula task are projected so as to have noeffect on the rates for the end effector tasks, and then added to theend effector task so as to generate the virtual or proposed solution:

{dot over (q)} _(v) ={dot over (q)}+P{dot over (q)} _(can).

In general, one may write any secondary task (such as the softwarecenter motion of the manipulator) so that its projected Jacobian hasauthority only over those linear combinations of joints that are in thenull space of the primary task (such as J_(eff)) by projecting thesecondary task Jacobian (in our example J_(can)) through the projector Pprior to finding the secondary task manipulator movements ({dot over(q)}_(can)). These calculations may be made recursive, and additionaldetails regarding applicable mathematical calculation techniques havebeen described by Sicilliano and Slotine.

In addition to the specific projector method examples described above,similar projectors may be employed for a wide variety of uses withinrobotic and/or surgical master-slave controllers. The projectorsdeveloped above are powerful and may be used throughout the controllerto give a directed mechanism for controlling certain degrees of freedomin certain ways. For example, to the right of “J⁻¹” in the block diagramof FIG. 12 is the bulk of the joint controller system 558. Using theabove techniques, we could replicate that bulk, yielding two controllerswith the desired or commanded velocity fed into each. Let us furtherfilter, or project, the desired velocity according to task. We couldthus create a controller for each task, each with its own set of limits,gains, saturations, and the like. In this manner it is possible to makedifferent tasks lose priority or give up under appropriate conditions.

Still further secondary tasks may be performed using projection,augmentation, and/or other methods, including some or all of the roboticmanipulations and/or set up methods described above. The generation ofsecondary tasks to perform collision avoidance described herein andelsewhere in the literature (see Sicilliano and Slotine and referencescited thereby) can, for example, be revised so as to inhibit collisionsbetween two simultaneously moving manipulator assemblies. One solutionwill use the general formalism of augmented tasks to generate ratecontrollers, as discussed above.

Additional modified control approaches may be implemented for otheroperation modes. For example, to lock an arm or manipulator assembly, ora subset of the degrees of freedom of such a structure (see clutchingbelow), we can deliberately clamp some or all of the output vector ofthe first module 544 to zero. To re-establish master-slave following, werelease the clamp, preferably after satisfying elementary matchingconditions to rule out jump discontinuities in the input command.

Using the control algorithms described above, we can also now understandhow to clutch a subset of joints or a subspace of the velocity space soas to provide an instrument clutch and/or a port clutch mode. In aminimally invasive robotic surgical system, an instrument clutch modewill generally allow manual articulation of the manipulator assemblywith limited degrees of freedom. For example, when removing andreplacing a first instrument with a second instrument having a differentsurgical end effector, an assistant may want to be able to manually movethe manipulator assembly so as to advance and retract the instrumentthrough the access site, to pivot the instrument about the access siteto orient and position the new end effector, and the like. These desiredclutch degrees of freedom will often not be aligned with the degrees offreedom of the mechanical joints of the system, so that simply releasingone or more joints might allow undesired or even dangerous movement ofthe manipulator system. Such instrument replacement clutching might befacilitated, however, if the system inhibited mathematically describedmotions of the manipulator assembly, such as lateral movement of a shaftor canula inserted into the access site by generating and using aprojector matrix or one of the augmented solutions described above insecond module 546 so as to drive one or more joints in response tomanual movement of the manipulator assembly. Similarly, a port clutchmode may allow the canula or access site to be moved (and optionally toidentify the new access site location when the move is completed), butmay maintain an orientation and/or pose of some or all of themanipulator assembly during manual movement.

If the clutched degrees of freedom of the slave manipulator linkagecoincide with one or more joint degrees of freedom (that is, if somejoints are locked and some joints are free to move in the clutch mode),then clutching is direct: one simply turns off the controller for thosejoints that are free to move. However, it will often be advantageous toclutch joints in a dependent way, where motion of one joint is linked bythe controller to motion of at least one other joint so that they can bemanually articulated together as a single degree of freedom. This may beachieved by driving at least one joint of a robotic manipulator assemblyin response to external articulation of at least one other joint. Thecontroller can effect this motion, which will often be different thanany degree of freedom of the mechanical system, by defining any desiredarbitrary linear combination of rates that can be treated as a singledegree of freedom that the operator may manipulate, optionally whilesome or all of the other mechanical degrees of freedom remain locked.This general concept includes port clutching, instrument clutching,elbow clutching (in which an intermediate elbow is allowed to move, forexample, from an upward oriented apex configuration around to alaterally oriented apex configuration while movement at the end effectorremains inhibited), and other clutching modes. Normal joint-by-jointclutching falls out as a special case.

Giving one example of a controller system to effect clutching that canagain be understood with reference to FIG. 12, at the beginning of theclutch, the virtual input command x_(v) is tracking x_(m) and the armappears locked. As the system switches into clutch mode, we create aprojector P that describes the linear combination of joint velocitiesthat should not be free to move:

P _(lock) =J _(lock) ^(#) J _(lock) ;P _(free)=1−P _(lock).

If this projector is inserted in the path after block “544 (J⁻¹)” in theblock diagram, then those velocities are hidden from the controller.

If such a projector is implemented while the arm is following, the armwill still attempt to follow but that linear combination of joints willnot have a command. Moreover, should that linear combination of jointshappen to move (as back-driven by the operator or the like), thecontroller will be blind to it and will not correct it.

Still further tasks or goals of the system may be implemented using thecontroller techniques described herein. Adaptive filtering and learningof a pivotal access site location from instrument movements aredescribed below. Additionally, haptic constraints may be implementedusing another variation on these techniques.

Regarding haptic constraints, it is often useful to provide cues to theoperator during the manipulation of the slave arm via the master inputdevice. For example, it may be desirable to indicate to the operator tokeep away from suboptimal configurations that will cause controllabilityproblems later. One simple method to provide such haptic constraints isto implement a joint limit technique. Another method is to turn on aone-sided servo whenever a threshold is crossed. This can be done usingany of the projector or augmentation methods above.

A one-sided servo would often push back toward or away from a controlpoint if the sensors indicate that the threshold is crossed and motionis further in the wrong direction. A velocity that is parallel to theboundary can be described for every point along the boundary. A dotproduct of the velocity vector and the threshold then indicates whetherthe operator is moving toward or away from the boundary, and theprojection in the wrong direction is resisted. This can all be done withthe above-described projector operators and a comparison with zero.

Referring now to FIG. 13, a software center surgical robotic manipulatorassembly 560 has particular advantages relating to movement of a pivotalcenter of motion 562 of a robotic instrument 564 with movement of anassociated port site or minimally invasive surgical access into apatient 566. Manipulator assembly 560 may be mounted to a patient sidetable, ceiling mount or floor mount, and can compensate for port sitemotion (e.g., patient breathing) by independently controlling thelocation of the port site. In the exemplary embodiment of FIG. 13, theport site location can be controlled in response to Cartesian forceinformation at the cannula pivotal center point as sensed by a forcesensing cannula 568.

The coordinate frame attached to the cannula is designated Ocan in FIG.13. This frame is distinct from a base frame, Obase, at the base of themanipulator and the tip frame, Otip, at the tip of the instrument. TheCartesian forces on the cannula can be resolved to control the positionof the cannula (Ocan). The torques about Ocan are not needed for suchposition control. Some or all of the forces at the port can be resolvedusing a force sensing system of cannula 568, and/or at least some of theforces may be resolved using a force sensing system of the instrument564, the manipulator assembly 560 (e.g., joint torque sensors of themanipulator), or the like.

Given the manipulator assembly geometry and position and sensing ofCannula forces, control laws can move the cannula position in responseto reaction forces at the cannula. One example of such a control law isto set a commanded cannula Cartesian force to zero. The controller willthen try to move the manipulator assembly such that the resultingCartesian force on the cannula is minimized. It is also possible to seta nonzero commanded cannula force, e.g., 5 lbs in the positive Z cannuladirection to have the arm help retract the cannula from the patient.This can also be achieved using the same controller.

A representative controller 570 for driving the manipulator assembly 560in response to motion of the minimally invasive access site is shown inFIGS. 13A-13C. FIG. 13A shows the overall block diagram for the cannulamotion controller as it would fit into an overall software centertele-operation controller. Signals from force sensors 572 can be derivedfrom the sensor characteristics and their placement in the arm. Atransform module 574 (typically a Jacobian operator) transforms measuredforce sensor information into the reference frame of Cartesian cannulaforces. The commanded cannula force 576 can be provided parametrically(e.g., as a programmed value). In the case of breathing compensation,this commanded force might, for example, be zero (fx=0, fy=0, fz=0). Inthe case of retraction, it might be (fx=0, fy=0, fz=a) where a is aretraction value (e.g., in Newtons). Filter module 578 creates afiltered error signal from the actual cannula force and the commandedcannula force. The commanded slave tip position 580 comes from themaster side of the teleoperation loop (see, for example, FIG. 11) and acommanded slave tip velocity 582 may also be provided.

Hybrid controller module 584 accepts command and actual slave positionand velocity signals, along with filtered cannula force error signals,and creates joint motor torque signals. The dynamics of the softwarecenter manipulator assembly are shown as 586, which interact with thedynamics of the patient environment 588.

FIGS. 13B and 13C show representative control arrangements or algorithmsfor filter module 578 and hybrid controller 584, respectively. In thecase of the filtered force command, a simple compare module 590 cancreate an error signal between commanded and actual cannula force. Thefiltered error can then be generated by a low pass filter 592. In thecase of the hybrid controller 584, a good method of control may be toconvert the commanded cannula force into a commanded cannula motion 594.This can be done by creating an incremental commanded positionproportional to the cannula force error and adding this incrementalcannula position to the current (actual) cannula position to create acommanded cannula position. The commanded cannula position is vectorassembled 596 with the commanded slave tip position, to create a commandposition vector for the manipulator assembly. This is combined withactual joint positions and velocities and fed into the software centerslave controller 598 to create commanded motor torques, which are inturn used to drive the manipulator assembly motors.

Reference is now made to FIGS. 14A and 14B which illustrate aninstrument coordinate frame and an instrument shaft moving pivotallyabout a minimally invasive access site, respectively. As shown in FIG.14A, pivot point PP is the position relative to the minimally invasiveaccess port at which an instrument shaft pivots. Pivot point PP may, forexample, be calculated by initially determining an original position ofthe interface between the manipulator instrument 14.1 and a unit vectorUo which has the same orientation as the instrument shaft. The pivotpoint position PP(x,y,z) values can be derived from various sensors ofthe manipulator assembly.

Referring to FIG. 14A the instrument can be within a first coordinateframe (x,y,z) which has the angles θ4 and θ5. The unit vector Uo can becomputed by the transformation matrix:

${Uo} = {\begin{bmatrix}{\cos \mspace{11mu} \Theta_{5}} & 0 & {{- \sin}\mspace{11mu} \Theta_{5}} \\{{- \sin}\mspace{11mu} \Theta_{4}\sin \mspace{11mu} \Theta_{5}} & {\cos \mspace{11mu} \Theta_{4}} & {{- \sin}\mspace{11mu} \Theta_{4}\cos \mspace{11mu} \Theta_{5}} \\{\cos \mspace{11mu} \Theta_{4}\sin \mspace{11mu} \Theta_{5}} & {\sin \mspace{11mu} \Theta_{4}} & {\cos \mspace{11mu} \Theta_{4}}\end{bmatrix}\begin{bmatrix}0 \\0 \\{- 1}\end{bmatrix}}$

After each movement of the end effector an angular movement of theinstrument ΔΘ can be computed by taking the arcsin of the cross-productof the first and second unit vectors Uo and U1 of the instrument inaccordance with the following line equations Lo and L1.

ΔΘ=arcsin([T])

T=Uo×U1

where:T=a vector which is a cross-product of unit vectors Uo and U1.

The unit vector of the new instrument position U1 may again bedetermined using the position sensors and the transformation matrixdescribed above. If the angle Δθ is greater than a threshold value, thena new pivot point may be calculated and Uo set to U1. As shown in FIG.14B, the first and second instrument orientations can be defined by theline equations Lo and L1:

Lo: xo=Mx0·Zo+Cxo

yo=Myo·Zo+Cyo

L1: x1=Mx1·Z1+Cx1

y1=My1·Z1+Cy1

where:Zo=a Z coordinate along the line Lo relative to the z axis of the firstcoordinate system.Z1=a Z coordinate along the line L1 relative to the z axis of the firstcoordinate system.Mxo=a slope of the line Lo as a function of Zo.Myo=a slope of the line Lo as a function of Zo.Mx1=a slope of the line L1 as a function of Z1.My1=a slope of the line L1 as a function of Z1.Cxo=a constant which represents the intersection of the line Lo and thex axis of the first coordinate system.Cyo=a constant which represents the intersection of the line Lo and they axis of the first coordinate system.Cx1=a constant which represents the intersection of the line L1 and thex axis of the first coordinate system.Cy1=a constant which represents the intersection of the line L1 and they axis of the first coordinate system.

The slopes may be computed using the following algorithms:

Mxo=Uxo/Uzo

Myo=Uyo/Uzo

Mx1=Ux1/Uz1

My1=Uy1/Uz1

Cx0=Pox−Mx1·Poz

Cy0=Poy−My1·Poz

Cx1=P1x−Mx1·P1z

Cy1=P1y−My1·P1z

where:Uo (x, y and z)=the unit vectors of the instrument in the first positionwithin the first coordinate system.U1 (x, y and z)=the unit vectors of the instrument in the secondposition within the first coordinate system.Po (x, y and z)=the coordinates of the intersection of the end effectorand the instrument in the first position within the first coordinatesystem.P1 (x, y and z)=the coordinates of the intersection of the end effectorand the instrument in the second position within the first coordinatesystem.

To find an approximate pivot point location PP, the pivot points of theinstrument in the first orientation Lo (pivot point Ro) and in thesecond orientation L1 (pivot point R1) can be determined, and thedistance half way between the two points Ro and R1 is computed andstored as the pivot point R_(ave) of the instrument. The pivot pointR_(ave) is determined by using the cross-product vector T.

To find the points Ro and R1 the following equalities can be set todefine a line with the same orientation as the vector T that passesthrough both Lo and L1.

tx=Tx/Tz

ty=Ty/Tz

where:tx=the slope of a line defined by vector T relative to the Z-x plane ofthe first coordinate system.ty=the slope of a line defined by vector T relative to the Z-y plane ofthe first coordinate system.Tx=the x component of the vector T.Ty=the y component of the vector T.Tz=the z component of the vector T.

Picking two points to determine the slopes Tx, Ty and Tz (e.g. Tx=X1−xo,Ty=y1−yo and Tz=z1−z0) and substituting the line equations Lo and L1,provides a solution for the point coordinates for Ro (xo, yo, zo) and R1(x1, y1, z1) as follows.

zo=((Mx1−tx)z1+Cx1−Cxo)/(Mxo−tx))

z1=((Cy1−Cyo)(Mxo−tx)−(Cx1−Cxo)(Myo−ty))((Myo−ty)(Mx1−tx)−(My1−ty)(Mxo−tx))

yo=Myo·zo+Cyo

y1=My1·z1+Cy1

xo=Mxo·zo+Cxo

x1−Mx1·z1+Cx1

The average distance between the pivot points Ro and R1 can be computedwith the following equation and can be stored as the pivot point of theinstrument.

R _(ave)=((x1+x0)/2,(y1+yo)/2,(z1+zo)/2

The pivot point location can be regularly continually updated with theabove described calculations.

Referring now to FIGS. 14A, 14B, and 15, another control systemschematic 400 illustrates how a movement of the master controller ormanipulator can be used to calculate a corresponding movement of asurgical instrument so that the instrument pivots about a pivot pointusing an alternative controller architecture. In accordance with thecontrol system shown in FIG. 15, the processor computes output signalsfor the manipulators so that the instruments move in conjunction andcoordination with the movements of the master input handle. Many aspectsof control system 400 are more fully described in U.S. Pat. No.6,699,177, the full disclosure of which is incorporated herein byreference. In the control system 400 illustrated in FIG. 15, thelocation of a pivot point at which an instrument is inserted into aninternal surgical site can be calculated by a pivot point calculationmodule 404 from instrument movements. The various modules shown in FIGS.15 and 16 may again comprise hardware, software, firmware, a combinationof hardware and software, or the like. The modules will often comprisedata processing hardware configured to perform the calculationsdescribed herein using software, and the calculations of multiplemodules may be integrated together in a common program or module, and/orthe calculation of a single module may be separated into multipleprograms or subroutines to be run on separate processor boards.

In short, the above descriptions enable the pivot (center) point to bedetermined/estimated through software. By having the capability tocompute software pivot points, different modes characterized by thecompliance or stiffness of the system can be selectively implemented.More particularly, different system modes over a range of pivotpoints/centers (i.e., ranging from one have a passive pivot point to onehaving a fixed/rigid pivot point) can be implemented after an estimatepivot point is computed. For example, in a fixed pivot implementation,the estimated pivot point can be compared to a desired pivot point togenerate an error output which can be used to drive the instrument'spivot to the desired location. Conversely, in a passive pivotimplementation, while the a desired pivot location may not be anoverriding objective, an estimated pivot point can be used for errordetection and consequently safety because changes in estimated pivotpoint locations may indicate that the patient has been moved or a sensoris malfunctioning thereby giving the system an opportunity to takecorrective action.

The interaction between the moving instrument and the tissue of theminimally invasive aperture may be determined at least in part by theprocessor, the processor optionally allowing the compliance or stiffnessof the system to be changed throughout a range extending from a passivepivot point to a fixed pivot point. At the passive end of thepassive/rigid range, the proximal end of the instrument may be moved inspace while the motors of the instrument holder wrist joint apply littleor no torque, so that the instrument acts effectively like it is coupledto the manipulator or robotic arm by a pair of passive joints. In thismode, the interaction between the instrument shaft and the tissue alongthe minimally invasive aperture induces the pivotal motion of theinstrument about the pivot point PP. If the surgical instrument was notinserted into the minimally invasive aperture or otherwise constrained,it may point downward under the influence of gravity, and movement ofthe manipulator arm would translate the hanging instrument withoutpivotal motion about a site along the instrument shaft. Toward the rigidend of the passive/rigid range, the location of the minimally invasiveaperture may be input or calculated as a fixed point in space. Themotors associated with each joint of the kinematic chain disposedproximal of the pivot point may then drive the manipulator so that anylateral force laterally against the shaft at the calculate pivot pointresults in a reaction force to keep the shaft through the pivot point.Such a system may, in some ways, behave similar to mechanicallyconstrained remote center linkages. Many embodiments will fall betweenthese two extremes, providing calculated motion which generally pivotsat the access site, and which adapts or moves the pivotal center ofmotion within an acceptable range when the tissue along the minimallyinvasive access site moves, without imposing excessive lateral forces onthat tissue.

It should be understood that many of the calculations described hereinregarding the controller architecture of FIG. 14 may, at least in part,be interchangeable with the controller calculations described aboveregarding alternative control architectures. For example, a softwarecenter manipulator assembly controller such as that illustrated in FIG.12 can leave the software center rather soft or lightly held so that thetissue forces applied by the interaction with the patient are allowed tooverpower the commands sent to the manipulator motors. Such calculatedsaturation of the manipulator drive torques may be allowed while stillcontrolling the end effector position to quite tight tolerances, sinceall motions of the manipulator assembly can be determined from thesensors of the manipulator. Several mechanisms may be applied forgradually increasing the stiffness of the software center control,including the weighted augmentation solutions discussed above. Bystarting with a small weighting value for the remote center, and bygradually increasing those weights to be on a par with the end effectorweights, one could (for example) gradually stiffen control over thepivotal center location.

The body wall and any other exogenous forces applied to the manipulatorassembly can affect the motion of the pivotal center if the pivotalcenter is held softly by the controller. The above-describedcalculations for determining the remote center location can be appliedif the instrument is allowed to passively pivot adjacent the interfacewith the manipulator, for example. As the variance of the estimates forthe pivotal motion center decreases, the weighting factor or gain may beincreased. Tracking of the changes in pivotal center location may allowidentification of and compensation for cyclical physiological movementssuch as patient breathing, beating of the heart, or the like. In someembodiments, similar calculations will allow the controller tocompensate for movement of the patient such as by reorienting a surgicaltable, repositioning a patient on the surgical table, or the like.

To provide a calculated pivotal motion of an instrument in space,regardless of whether or not the instrument is inserted through aminimally invasive aperture, the systems described herein may optionallyinclude a modified manipulator command module 406 of controller 400having a software center joint command calculation module 410 asillustrated in FIG. 16. Software center module 410 of processor 210receives an input command corresponding to a desired end effectormotion, typically in the form of signal generated in response to amaster input handle movement. Software center module 410 calculates thedesired corresponding end effector movement or vector at calculationmodule 412, optionally using coordinate system transformations such asthose described in U.S. Pat. Nos. 6,424,885 and/or 6,699,177. Thedesired end effector movement is then transmitted to an inversekinematics solver 414. The desired end effector motion is also used by aremote center locator 416 so as to generate a solved center about whichthe instrument will pivot.

Along with the desired end effector motion, the remote center locator416 makes use of actual joint motion information from the slavemanipulator 418. The information from manipulator 418 may compriseactual joint motions from prior end effector movements. The data mayoriginate in the form of positions of each of the individual joints frompotentiometers or the like, discrete steps of the individual joints fromencoders or the like, or a wide variety of alternative motionindicators. Optionally, joint torques as detected at the joints of themanipulator may also be provided, and both joint positions and jointvelocities may be transmitted to a forward kinematics module 420. Theforward kinematics module 420 can calculate an observed minimallyinvasive aperture pivotal center location from the behavior of the slavemanipulator during prior movements of the instrument.

Along with motion about a pivot point, the software center module 410may also determine forces associated with the pivoting about theminimally invasive aperture, properties of the tissue tract (forexample, the lateral or pivotal spring forces applied by the tissuetract to the instrument during movement of the instrument shaft and/orany lateral movement of the patient anatomy adjacent the aperture) andthe like. Still further, properties of the observed pivotal center mayalso be determined. For example, the software center module 410 maydetermine any cyclical movements of the observed pivotal center with aphysiological activity such as breathing or beating of the heart.Shifting of the pivotal center with changing pivotal angles ororientations may also be identified. For example, intercostal access tothe chest cavity may result in a first pivotal center when a instrumentinserted between the ribs is pivoted generally across the riborientation, and a slightly different pivotal center when the instrumentis angled along the lengths of the adjacent ribs. Tissue stiffness mayalso vary along the axis of the tissue tract and/or at differingdistances from the tissue tract, so that pivoting of a shaft insertedthrough a minimally invasive aperture at a relatively small angle mayresult in a pivotal center which is disposed at a first point along atissue tract extending through the abdominal wall, while increasing theangle of the shaft inserted through the same tract may shift the pivotalcenter axially along the tissue tract either proximally or distally.

The observed pivotal center information from the forward kinematicsmodule 420 (along with any associated forces) may be combined with thecommanded end effector motion from desired end effector calculationmodule 412 so as to generate a solved or desired center of the desiredmotion. The observed pivotal center and commanded tip motion may becombined using an adaptive filter, and the input to the inversekinematic solver from the combination of the observed pivotal center andthe commanded end effector motion may comprise a desired remote centermotion 422.

In general, manipulator linkages which are not easily back driven willtend to keep a remote center location at a calculated position. Assumingthe drive system of the linkage is not overpowered, the observed remotecenter will remain within a close proximity to the desired or calculatedremote center location, with the difference between the observed centerand desired center often falling within a relatively small servo errorof the manipulator. However, the system may determine that the remotecenter should be removed to alleviate forces and/or torques applied tothe manipulator structure. For example, the forces and/or torques may beapplied to and measured at the cannula, for example, by reactive forcesof the tissue against the cannula, by physiological movement of thepatient during breathing or patient repositioning, or the like. In someembodiments, an operator may manually push on the arm to move the remotecenter to a new location, optionally before, during, or aftertransmitting a pivot center repositioning signal to the processor, suchas by pushing a port clutch button. As described above, a large numberof computational methods may be used to choose a new remote center basedon the data provided by such manual movement of the manipulator linkage.The output of such calculations may be termed a solved center or adesired center, and will often represent an input from a system user tothe processor, optionally by releasing the port clutch input when theappropriate structure along the kinematic chain of the manipulatorassembly (such as a cannula) is disposed at the desired access sitelocation.

In more easily back driven arms, a solved center or desired center maysimply be a filtered version of an observed center. Such back drivablelinkages lend themselves more directly to observing actual motions ofthe remote center and adjusting the control appropriately. As strongeror less back drivable manipulator linkages will tend to exhibit littlemotion of the observed center, employing a torque or force sensor tomeasure disturbance forces acting on the arm at the pivot center (suchas a cannula force sensor) may facilitate such pivot location movement,rather than relying solely on joint torque or position measurements.

As described above, the shaft of the instrument between the end effectorand the proximal instrument housing will typically extend through theminimally invasive aperture. The motion of this intermediate portion ofthe instrument will be calculated so that the instrument does not imposeinjury on the patient, but instead remains within the minimally invasiveaccess site. Along with pivotal motion of the instrument shaft, theinstrument may rotate about its shaft, and the shaft may be inserteddistally or withdrawn proximally along the shaft axis.

So as to allow inverse kinematic solver 414 to calculate the jointmotions to effect the commanded end effector movement while maintainingthe instrument within the minimally invasive aperture site, the vectorof the commanded Cartesian velocities and angular velocities of the tipof end effector may be provided from the slave vector calculator 414,the exemplary end effector vector comprising a 1×6 vector, withactuation of grip on end effector often being handled separately fromthe inverse kinematic computations. The desired remote center motion 422input into the inverse kinematic solver may comprise a vector ofCartesian velocities of the pivotal center, such as a 1×3 vector.Algorithms that compute movement of the pivotal center (for example,based on measuring of forces at the pivotal center of a high impedanceor rigidly driven arm) or the effective location of a pivotal center(based on observed pivotal motions using a low impedance or morepassively pivoting arm) can be adjusted using these vector inputs.Hence, inverse kinematic solver 414 can be used for manipulatorstructures which are a combination of a compliant or passive rotationalcenter and a more rigidly calculated rotational center.

As noted above, the manipulator/instrument linkages often have a greaternumber of degrees of freedom than the end effector has in the surgicalworkspace. Inverse kinematic solver 414 may nonetheless make use ofcontroller schemes similar to those of more constrained mechanicalsystems, such as those described in U.S. Pat. No. 6,493,608, the fulldisclosure of which is incorporated herein by reference. Morespecifically, the use of modified inverse kinematics with a feedbacktuned to correct for integrator drift may have some benefits for thejoint motion calculator 410 of FIG. 16. However, certain modificationsto the inverse kinematic solver 414 may be employed to adapt the jointcalculator for use with a larger number of degrees of freedom and/or forthe use of a calculated remote center.

Exemplary calculations performed by inverse kinematic solver 414 areillustrated in more detail in FIG. 17. Using the desired end effectormotion and the desired pivotal center motion, inverse kinematic solver414 first determines velocities of the instrument. In the exemplaryembodiment the velocities will include a vector of the desired Cartesianvelocities of the end effector, the desired angular velocities of theend effector, and the Cartesian velocities of the pivotal center, oftenin the form of a 1×9 vector V. From this vector, the vector of jointvelocities that will cause the end effector tip and the intermediateinstrument shaft adjacent the aperture to achieve the desired Cartesianand angular velocities, {dot over (q)}_(base)) may be computed asfollows:

{dot over (q)} _(base) =W ⁻¹ J ^(T)(JW ⁻¹ J ^(T))⁻¹ V

in which W is a weighting matrix and J is the Jacobian matrix. For theexemplary kinematic chain illustrated in FIG. 6, the Jacobian matrix isa 9×10 matrix.

The exemplary arms will again often include more degrees of freedom thanthe combination of the degrees of freedom of the end effector in space(often 6) and the constraint of maintaining the instrument shaft at theaperture (3), so that they are underconstrained for the end effector andshaft pivoting tasks. Hence, these linkage systems may have more than 10degrees of freedom, and their kinematic chains often define a null spaceor range of alternative configurations and/or joint velocities which thekinematic chain can have for a given end effector position. A projectorvector P into the null space of the Jacobian matrix can be calculated asfollows:

P=(I−J _(rpi) *J)

in which I is the identity matrix and J_(rpi) is the right pseudoinverse of the Jacobian matrix.

A desired set of auxiliary joint velocities q_(aux) that minimizes oneor more cost functions can then be computed, with the cost functionoften having a configuration which drives the manipulator away fromjoint range-of-motion limitations, away from singularities, free fromobstacles, and the like. These auxiliary joint velocities can then beprojected into the null space of the Jacobian matrix. The final velocityvector may then be calculated by adding the base and auxiliary jointvelocity vectors together as follows:

{dot over (q)}={dot over (q)} _(base) +P*{dot over (q)} _(aux)

Referring once again to FIG. 9, kinematic solver 414 can calculate thejoint motions based at least in part on information regarding themotions of other manipulators. In the exemplary embodiment, a poseoptimizer module 430 receives actual joint motion data from themanipulator, and also receives actual joint motion information from theother manipulators. Based on the motions of the manipulators, the poseoptimizer calculates additional inverse kinematic constraints to beinput into the inverse kinematic solver 414. The pose optimizer mayagain drive the joint configuration of the kinematic chain using a goalfunction, such as to enhance separation between adjacent manipulatorswhile effecting the commanded movement of the end effector, and whilealso maintaining an intermediate portion of the instrument within aminimally invasive aperture site. Pose optimizer 430 thereby takesadvantage of the excess degrees of freedom of the kinematic linkage(beyond those of the end effector in the surgical site), as can beunderstood with reference to FIGS. 5A and 5B.

In FIG. 5A, manipulator 304 holds instrument 306 at a given location inspace. An intermediate joint J4 of the manipulator is configured so thatthe axes of the links coupled together at joint J4 generally define adownward oriented apex 350. As manipulator 304 is pivotally mounted tothe base at a first pivotal joint 322, is pivotally mounted toinstrument holder 320 at an instrument wrist 332, and includes a rolljoint and two pivotal joints therebetween, manipulator arm can be drivenfrom this downward apex configuration to an upward oriented apexconfiguration illustrated in FIG. 5B, with the upward oriented apexindicated by reference numeral 352. Note that the end effector (and infact, the entire instrument) may be at the same location in spacerelative to the base supporting the manipulator when the manipulator isin either of these two configurations, and while the manipulator movesbetween these two configurations.

So as to maintain accurate control over the surgical instruments andavoid wear or damage to the system, it will often be advantageous toconfigure adjacent manipulators such that collisions between themanipulators are inhibited. For example, when two manipulators aresupported by one or more bases such that the manipulator arms areadjacent to each other, and when the surgical instruments supported bythe manipulators are inserted through adjacent minimally invasiveapertures, having the manipulators in similar configurations may lead toconfigurations when the manipulators move towards each other. By insteadhaving adjacent manipulators alternate between a downward oriented apexconfiguration such as that illustrated in FIG. 5A and an upward orientedapex configuration such as that illustrated in FIG. 5B, the overallconfiguration of the manipulator system may have fewer or no collisions.

Referring once again to FIG. 15, pose optimizer 430 and joint calculator410 may be implemented in different embodiments for a multiplemanipulator system. For example, a joint calculation and associated poseoptimization may be performed individually for each manipulator motion.The calculations will be performed iteratively for each manipulator, andthe joint calculations may be performed by a single program running on asingle processor board, by separate programs running on dedicatedprocessor boards, or the like. Regardless, at least some of the poseoptimization program may run a pose optimization routine specificallyfor a single manipulator. Alternatively, a single global pose optimizermay take observed slave motion data from a plurality of manipulators,optionally from all of the manipulators in the manipulator system. Fromthis information, a globally optimized solution for the manipulator posemay be computed. This globally optimized solution may then be used as aninput for the inverse kinematic solver 414 for each of the manipulators.Still further alternatives are possible, including using inversekinematic solver to simultaneously determine actual joint motions of alljoints throughout the manipulator arms.

Referring now to FIG. 18A, for utility of a manipulator assembly in amultiple manipulator surgical system (see FIG. 1B, for example) eachmanipulator assembly should be compact and highly dexterous. Compactsize enables multiple arms to share a limited space available in avicinity of a patient while minimizing collisions between manipulatorassemblies during a surgical procedure. Compact size also reduces thetorque and power specifications of the motors and actuators, and helpsmake the system more safe for surgical staff and the patient. Compactmanipulator assembly structures also tend to have lighter weight, whichreduces the size of the manipulator support and setup system, and canmake a manual setup process easier for a surgical staff High dexteritygenerally allows the surgical end effector to be located, oriented, andmoved within the patient's body in the most clinically advantageousmanner.

The kinematic structure of a manipulator assembly can affect thecompactness and dexterity to a significant degree. FIG. 18A illustratesan exemplary kinematic structure which provides both compactness andhigh dexterity. These benefits are achieved by consideration of avariety of kinematic relationships and constraints, including balancingof primary segment lengths, reducing lateral kinematic offsets, andproviding at least a minimum range of motion for selected joints. Thesethree criteria will be described in more detail in the followingparagraph.

Regarding the segment for portion lengths of the manipulator assembly,and referring to FIGS. 18A, 4A-4C, 5A and 5B, and 6, the exemplarymanipulator assembly 304 includes a kinematic chain of links and jointsdefining two arm portions. A first arm portion extends between pivotaljoint J2 and pivotal joint J4 and has a length of L1. A second armportion extends between pivotal joint J4 and pivotal joint j6 and has alength of L2. Lengths L1 and L2 of these first and second arm portions602, 604 each extend along an associated axis of the associated armportion, and a roll joint is present along each arm portion (see jointsj3 and j5 in FIG. 6).

The useful work volume of manipulator assembly 304 is generally improvedby keeping lengths L1 and L2 within about 20% of one another. If theselengths differ significantly more than that amount, volumetric holes orunreachable areas in the work volume can become problematic. Balancinglengths L1 and L2 also help to balance the angular range of motion fordriving the joints J2 and J4. As the mechanical design of the jointsgets significantly more complex when the range of motion increasesexcessively, it is generally beneficial to avoid designs involving verylarge ranges of motions about these pivotal joints.

Next addressing the lateral kinematic offsets, and referring here toFIGS. 18B, 18C, and 18D, the kinematic, structural, and mechanicaldesign of manipulator 304 as seen in these top views presents a slenderprofile with a limited width as measured from a central plane PL. In theexemplary embodiment a distal width W_(d) is less than a proximal widthW_(P). Although manipulator assembly 304 will move from the planarconfiguration during robotic operation, it will often be advantageous todrive the linkages so as to maintain a substantially planar profilealong plane PL, and/or to minimize the protrusions or widths of the armfrom the central plane in a direction of an adjacent manipulatorassembly. Such limiting of lateral kinematic offsets in the arm designand/or drive control logic may avoid mechanical protrusions or bumpswhich can potentially collide with other manipulator assemblies, thesetup and manipulator support structure, the patient, surgical staff,and the like. Reducing offsets from plane PL may also help avoid holesor unreachable spaces in the arm work volume. Driving the manipulator toavoid unnecessary lateral offsets from a planar configuration can beachieved by the augmented and projector solutions described above, withthis constraint or task often being lower in priority that end effectorfollowing, pivoting about the cannula, and the like.

In the exemplary embodiment illustrated in FIG. 18B (and referring tothe joints and their associated axes illustrated in FIG. 6), when thearm is in a neutral pose, the pivotal axis of joint 1, the pivotal axisof joint 3, the pivotal axis of joint 5, and the linear axis of joint 7,all lie along plane PL. This condition changes as the arm moves, but inthe exemplary embodiment, axes of joints J3 and J5 remain co-planar. Inother words, the offsets in the kinematic chain along these jointsshould ideally be zero by design.

As illustrated in FIGS. 18C and 18D, the use of adjacent arms 304 a and304 b which are designed for use along an associated plane PL 1 and PL2,when driven by controllers having a goal of maintaining a planarconfiguration of each arm at planes which are laterally and/or angularlyoffset, can inhibit collisions between these adjacent arms. As notedabove, alternating adjacent arms between apex upward and apex downwardconfigurations may also be employed, as described above.

Regarding the minimum ranges of motion for the joints of the manipulatorassembly, range of motion may be enhanced by providing at least one ofthe minimum ranges of motion in the following table for the associatedjoint, preferably some of the ranges of motions in this table, andideally all of the minimum ranges of motions (“ROM”) in the followingtable:

Joint Min ROM 1  90° 2 135° 3 180° 4 180° 5 180° 6  90° 7 8 inches

Particular benefit in the overall dexterity of the highly configurablemanipulator assembly 304 is provided by attention to joints J3,sometimes referred to as upper arm roll, and J6, which is sometimesreferred to as wrist pitch. In many ways, the upper arm roll joint J3can be thought of as the redundant or extra degree of freedom of thearm. Without it, manipulator assembly 304 would work very effectively inthree-dimensional space. By providing this additional degree of freedom,the arm is able to attain an infinite number of poses or configurationsfor a given end effector position, even while the shaft of theinstrument remains constrained to pivotal motion about a minimallyinvasive aperture. This can allow the arm to avoid obstacles, avoidcollisions with other arms, and the like. This flexibility also can beused to optimize the kinematics of the manipulator linkage so as toavoid joint limits, singularities, and the like.

When used for robotic surgical systems having a plurality of manipulatorassemblies, the manipulator linkage 304 will ideally be able to invertitself from an apex oriented downward configuration to an apex orientedupward configuration and back, as illustrated in FIGS. 5A and 5B. Thiscan enable adjacent arms to alternate between these two configurationsin primary pose (e.g., nominal starting pose), giving each arm a largerobstacle-free workspace in which to move, for example. This ability toinvert the arm is facilitated by providing upper arm roll joint J3 withat least 180 degrees of travel, preferably with 360 degrees or more oftravel.

Referring now to FIGS. 18E and 18F, the ability to invert the armbetween apex oriented upward and apex oriented downward configurationsis also facilitated by providing the wrist pitch joint J6 with a rangeof motion sufficient to allow the instrument axis 612 to move quiteclose to the adjacent distal primary axis 614 of manipulator assembly304, ideally in both directions. In other words, angle J6A when the endeffector of the instrument is as close as possible to the apex of themanipulator arm, will preferably be 45 degrees or less, while angle J6Bwill preferably be 135 degrees or more when the end effector is moved asfar away as possible from the apex of arm 304.

Referring now to FIG. 18G, it may be desirable to install collisionsensors 622 at one or more locations on the manipulator assembly 304.The sensors will often be disposed on the manipulator assembly's outersurfaces. The sensors may comprise contact and/or proximity sensors, andmay be monitored on a real time or near real time basis by thecontroller. In the event of arm-to-arm contact (or near contact), thecontrol system may take appropriate action such as driving themanipulator to a different pose, generating a soft fault, generating ahard fault, or simply generating some type of alert, such as an audiblebeep, a warning light, or the like to the system operator. In someembodiments, the appropriate action may depend on the location of thesensor and/or other characteristics of the collision detection signal.

Suitable collision sensors may employ tape switches such as thosecommercially available from Tape Switch Corporation of New York;flexible sensing arrays such as those developed by the University ofTokyo (including Takao Someya), and described in an article entitled“Flexible Sensors Make Robot Skin” in www.trnmag.com on Sep. 22/29,2004; individual force sensing resistors or force sensing resistorarrays available commercially from Interlink Electronics; capaciflectoractive or passive capacitive sensing systems such as those developed byJohn Vranish at NASA-Goddard, or the like.

While the exemplary embodiments have been described in some detail forclarity of understanding and by way of example, a variety ofadaptations, modifications, and changes will be obvious to those ofskill in the art. Hence, the scope of the present invention is limitedsolely by the appended claims.

What is claimed is:
 1. A medical system comprising: a manipulatorassembly having a plurality of links, the plurality of links extendingbetween a distal end and a proximal base and interconnected by aplurality of joints, the manipulator assembly being configured formoving the distal end relative to the proximal base; a first input forreceiving a movement command to effect a desired movement of the distalend; a second input for receiving a movement command to effect a desiredmovement of the manipulator assembly; and a processor coupling themanipulator assembly to each of the first input and the second input,wherein the processor is configured for determining calculated jointmovements that: effect the desired movement of the distal end inresponse to the first input, and effect the desired movement of themanipulator assembly in response to the second input.
 2. The medicalsystem of claim 1 wherein the processor is configured to effect thedesired movement of the manipulator assembly concurrent with the desiredmovement of the distal end.
 3. The medical system of claim 2 wherein thedesired movement of the distal end comprises a primary task and thedesired movement of the manipulator assembly comprises one or moresecondary tasks, wherein the processor is further configured todetermine the calculated joint movement according to a task priority orby scaling.
 4. The medical system of claim 1 wherein the processor isconfigured to effect the desired movement of the manipulator assembly ata different time from the desired movement of the distal end.
 5. Themedical system of claim 4 wherein the processor is configured with afirst and second mode of operation, the desired movement of the distalend being effected in the first mode of operation and the desiredmovement of the manipulator assembly being effected in the second modeof operation different from the first mode of operation.
 6. The medicalsystem of claim 5 wherein the first mode of operation is a tissuemanipulator mode; and the second mode of operation includes one or moreclutch modes, the one or more clutch modes including: a pose-clutchmode, a port-clutch mode, an instrument clutch mode, or any combinationthereof.
 7. The medical system of claim 1 wherein the first input isdisposed on an operator workstation.
 8. The medical system of claim 7wherein the first input comprises at least two control devicesconfigured to be manipulated while concurrently gripped by an operator.9. The medical system of claim 1 wherein the second input comprises asensor capable of detecting an external manual articulation of one ormore links of the manipulator assembly.
 10. The medical system of claim1 wherein the second input comprises one or more input devices disposedon the manipulator assembly.
 11. The medical system of claim 10 whereinthe one or more input devices are disposed on one or more links of theplurality of links.
 12. The medical system of claim 1 wherein themanipulator assembly is mounted or attached to a surgical table duringoperation.
 13. The medical system of claim 1, wherein the plurality ofjoints provide sufficient degrees of freedom between the proximal baseand the distal end to allow a range of joint states for a state of thedistal end.
 14. A medical system comprising: a manipulator assemblyhaving a plurality of links extending between a distal end and aproximal base and being interconnected by a plurality of joints, themanipulator assembly being configured for tele-operatively moving thedistal end relative to the proximal base; an input device for receivinga movement command to effect a desired movement of the manipulatorassembly, wherein the input device is disposed on the manipulatorassembly; and a processor coupling the manipulator assembly to theinput, wherein the processor is configured for determining jointmovements that effect the desired movement of the manipulator assemblyin response to the received movement command.
 15. A robotic method foroperating a manipulator assembly having a plurality of links extendingbetween a proximal base and a distal end, wherein the plurality of linksare interconnected by a plurality of joints, the method comprising:receiving from a first input a first operator command to move the distalend according to a desired movement of the distal end; roboticallymoving the distal end relative to the proximal base in response to thefirst operator command so as to effect the desired movement of thedistal end; receiving from a second input a second operator command tomove at least a portion of the manipulator assembly according to adesired movement of the manipulator assembly; and robotically moving themanipulator assembly in response to the second operator command so as toeffect the desired movement of the manipulator assembly.
 16. The roboticmethod of claim 15 wherein the first input is located at an operatorworkstation.
 17. The robotic method of claim 16 wherein the second inputcomprises a force or torque applied to the manipulator assembly by apatient side operator.
 18. The robotic method of claim 16, wherein thesecond input comprises an input device disposed on the manipulatorassembly.
 19. The robotic method of claim 15, wherein the desiredmovement of the manipulator assembly and the desired movement of thedistal end are effected at different times.
 20. The robotic method ofclaim 15, wherein the desired movement of the manipulator assembly iseffected concurrently with the desired movement of the distal end. 21.The robotic method of claim 20, wherein the desired movement of thedistal end comprises a primary task and the desired movement of themanipulator assembly comprises a secondary task, wherein the methodfurther comprises prioritizing or scaling the primary and secondarytask.
 22. The robotic method of claim 15, wherein the plurality ofjoints provide sufficient degrees of freedom between the proximal baseand the distal end to allow a range of joint states for a state of thedistal end.
 23. A robotic system comprising: a robotic manipulatorassembly comprising a tool holder and a sensor, wherein the tool holderis configured for receiving a tool to be manipulated by an operator, andwherein the sensor is configured to measure a torque or force acting onthe tool when supported in the tool holder; and a control systemcommunicatively coupled with the robotic manipulator assembly and thesensor, wherein the control system is configured to: communicate with anoutput device; receive an instruction from a user to switch between afirst operating mode and a second operating mode; and receive anindication of the torque or force acting on the tool and adjust anoutput presented by the output device based on the indication.
 24. Therobotic system of claim 23 wherein the control system is configured suchthat: in the first operating mode, the torque or force acting on thetool controls movement of the robotic manipulator assembly, and in thesecond operating mode, movement of the robotic manipulator assembly iscontrolled by an operator interface with the torque or force acting onthe tool being received as an input.
 25. The robotic system of claim 23wherein the control system is configured such that: the first operatingmode is a tissue manipulation mode; and the second operating modecomprises a pose clutch mode, a port-clutch mode, an instrument clutchmode or any combination thereof.
 26. A robotic system comprising: arobotic manipulator assembly; a control system communicatively coupledwith the robotic manipulator assembly; and an operator workstationcommunicatively coupled with the control system; wherein the controlsystem is configured to control a first portion of the roboticmanipulator assembly in response to a force or torque applied to asecond portion of the robotic manipulator assembly; and wherein thecontrol system is configured to control a third portion of the roboticmanipulator assembly in response to an input received from the operatorworkstation.
 27. The robotic system of claim 26 wherein the force ortorque is applied by manual external articulation of the second portionof the robotic manipulator assembly.
 28. The robotic system of claim 26wherein the first portion is a same portion as the third portion. 29.The robotic system of claim 28 wherein the first portion includes adistal link configured to support an instrument, wherein the controlsystem is configured such that control of the first portion is providedin an instrument clutch mode to facilitate exchange of an instrument bya patient side assistant and control of the third portion is provided ina tissue manipulation mode to facilitate control of the instrument by anoperator at the operator workstation.